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Logistics  Test  and  Evaluation  in  Flight  Test 

(RTO  AG-300  Volume  20  /  SCI-010) 


Executive  Summary 

Flight  testing  continues  to  remain  an  essential  step  in  the  development  or  modification  of  an  aircraft. 
Historically,  considerable  effort  and  expertise  have  been  given  to  the  operational  utility  of  a  system 
under  test  with  little  regard  to  supportability  test  and  evaluation,  except  in  areas  of  reliability  and 
maintainability  (R&M).  R&M  assessment  generally  is  measured  to  determine  specification  compliance 
without  a  thorough  assessment  of  the  opportunities  supportability  design  will  provide  in  reducing 
overall  system  costs.  As  we  test  future  aircraft  or  modifications  assessing  supportability  just  makes 
good  sense.  Sixty  percent  of  the  life  cycle  costs  of  systems  are  incurred  in  the  area  of  supportability, 
including  manpower,  maintenance  planning,  training  and  training  support,  support  equipment, 
facilities,  computer  resources,  design  interface,  supply  support,  technical  data,  and  packaging  storage 
and  transportation. 

The  purpose  of  this  AGARDograph  is  to  bring  attention  to  a  process  of  performing  logistics  test  and 
evaluation  (supportability  assessment)  while  conducting  flight  tests.  This  AGARDograph  presents  an 
approach  utilizing  maintenance  personnel,  R&M  engineers,  and  Human  Factors  (HF)  engineering  to 
document,  rate  and  evaluate  the  supportability  of  any  system  under  test.  The  consequences  can  be  very 
effective  in  decision  making  for  determining  manpower,  maintenance  planning  and  support  equipment 
purchases.  The  assessment  may  also  drive  system  design  changes  making  the  aircraft  more 
supportable.  All  this,  if  approached  correctly,  can  reduce  the  costs  of  sustainment  and  improve 
operational  availability. 
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les  Essais  et  revaluation  de  la  logistique 
lors  des  essais  en  vol 

(RTO  AG-300  Volume  20  /  SCI-010) 


Synthese 

Les  essais  en  vol  representent  toujours  une  etape  indispensable  dans  le  developpement  ou  la 
modification  d’un  aeronef.  Dans  le  passe,  des  efforts  importants  mettant  en  oeuvre  des  competences 
techniques  considerables  ont  ete  consacres  a  I’etude  de  I’interet  operationnel  des  systemes  a  I’essai, 
mais  sans  attacher  beaucoup  d’attention  a  revaluation  des  possibilites  de  soutien,  sauf  en  ce  qui 
concernait  la  fiabilite  et  la  maintenabilite  (R&M).  En  general,  revaluation  de  la  R&M  consiste  a 
determiner  la  conformite  aux  specifications  techniques  d’un  systeme  donne,  sans  evaluation  des 
possibilites  de  reduction  des  couts  globaux  des  systemes  qui  sont  offertes  par  une  conception 
permettant  d’augmenter  les  capacites  de  soutien.  II  nous  semble  logique  d’evaluer  les  capacites  de 
soutien  des  futurs  aeronefs  et  de  leurs  variantes.  Pres  de  soixante  pour  cent  des  couts  globaux  de 
possession  des  systemes  sont  consacres  a  la  capacite  de  soutien,  y  compris  la  main  d’oeuvre,  la 
planification  de  la  maintenance,  la  formation  et  le  soutien  de  la  formation,  les  equipements  de  soutien, 
les  installations,  les  moyens  informatiques,  les  interfaces  de  conception,  les  fournitures,  les  donnees 
techniques,  les  emballages,  I’emmagasinage  et  le  transport. 

Get  AGARDographe  a  pour  objectif  d’attirer  I’attention  sur  le  processus  des  essais  et  de  revaluation  de 
la  logistique  (evaluation  de  la  capacite  de  soutien)  lors  des  essais  en  vol.  II  presente  une  approche  qui 
associe  le  personnel  de  maintenance,  les  ingenieurs  R&M,  et  la  conception  ergonomique  (HE)  afin  de 
documentor  et  d’evaluer  la  capacite  de  soutien  de  tout  systeme  a  I’essai.  Les  resultats  peuvent  faciliter 
la  prise  de  decisions  concernant  la  main  d’oeuvre,  la  planification  de  la  maintenance  et  I’achat  de 
materiel  de  soutien.  L’evaluation  peut  aussi  servir  de  guide  aux  changements  qui  seraient  a  faire  au 
niveau  de  la  conception  des  systemes  pour  augmenter  la  capacite  de  soutien  des  aeronefs.  Moyennant 
une  mise  en  cEuvre  judicieuse,  1’ ensemble  de  ces  mesures  est  susceptible  de  reduire  les  couts  de  soutien 
et  de  permettre  une  plus  grande  disponibilite  operationnelle. 


IV 


Click  inside  the  blue  boxes  to  view  the  corresponding  section 


Contents 

Page 

Executive  Summary  iii 

Synthese  iv 

List  of  Figures  and  Tables  ix 

Preface  x 

Abstract  xi 

1.0  Introduction  1 

1 . 1  Purpose  1 

1 .2  Scope  1 

1.3  System  Description  1 

1.4  Document  Organization  1 

1.5  Test  Concept  1 

2.0  General  Information  about  the  Structure  and  Contents  of  Chapters  3-12,  5 

The  ILS  Elements 

2.1  Introduction  5 

2.2  General  Information  about  the  Disciplines  Referenced  in  Chapters  3-12,  The  ILS  Elements  8 

2.2.1  Logistics  Test  8 

2.2. 1.1  System  Specifications  and  Logistics  Test  8 

2. 2. 1.2  Logistics  Developmental  Test  and  Evaluation  (DT&E)  of  Data  9 

2.2.2  Reliability  and  Maintainability  9 

2.2.2. 1  System  RM(&A  Requirements  9 

2.2.3  Human  Factors  (HSI)  9 

2.2.3. 1  Weapon  System  Human  Factors  Requirements  9 

2.2. 3.2  Human  Factors  Developmental  Test  and  Evaluation  (DT&E)  Data  9 

3.0  Maintenance  Planning  (MP)  11 

3.1  Description  11 

3.2  Objective  11 

3.3  Logistics  Test  Measures  (LTMs)  11 

3.4  Reliability  and  Maintainability  11 

3.4.1  Evaluation  Measures  11 

3.4.2  Measurement  Criteria  13 

3.4.3  Analysis  Techniques  and  Tools  13 

3.5  Human  Factors  13 

3.6  Logistics  Test  13 

3.6.1  Evaluation  Measures  13 

3.6.2  Measurement  Criteria  14 

3.6.3  Analysis  Techniques  and  Tools  14 

3.7  Overall  Element  Analysis  14 

4.0  Manpower  and  Personnel  (M&P)  15 

4.1  Description  15 

4.2  Objective  15 

4.3  Logistics  Test  Measures  (LTMs)  15 

4.4  Reliability  and  Maintainability  15 

4.4.1  Evaluation  Measures  15 

4.4.2  Measurement  Criteria  16 

4.4.3  Analysis  Techniques  and  Tools  17 


V 


4.5 

Human  Factors 

17 

4.5.1  Evaluation  Measures 

17 

4.5.2  Measurement  Criteria 

17 

4.5.3  Analysis  Techniques  and  Tools 

17 

4.6 

Logistics 

17 

4.6.1  Evaluation  Measures 

17 

4.6.2  Measurement  Criteria 

18 

4.6.3  Analysis  Techniques  and  Tools 

18 

4.7 

Overall  Element  Analysis 

18 

5.0 

Support  Equipment  (SE) 

19 

5.1 

Description 

19 

5.2 

Objective 

19 

5.3 

Logistics  Test  Measures  (LTMs) 

19 

5.4 

Reliability  and  Maintainability 

19 

5.4.1  Evaluation  Measures 

19 

5.4.2  Measurement  Criteria 

19 

5.4.3  Analysis  Techniques  and  Tools 

20 

5.5 

Human  Factors 

20 

5.5.1  Evaluation  Measures 

20 

5.5.2  Measurement  Criteria 

20 

5.5.3  Analysis  Techniques  and  Tools 

20 

5.6 

Logistics 

20 

5.6.1  Evaluation  Measures 

20 

5.6.2  Evaluation  Criteria 

21 

5.6.3  Analysis  Techniques  and  Tools 

21 

5.7 

Overall  Element  Analysis 

21 

6.0 

Supply  Support  (SS) 

23 

6.1 

Description 

23 

6.2 

Objective 

23 

6.3 

Logistics  Test  Measures  (LTMs) 

23 

6.4 

Reliability  and  Maintainability 

23 

6.4.1  Evaluation  Measures 

23 

6.4.2  Measurement  Criteria 

23 

6.4.3  Analysis  Techniques  and  Tools 

23 

6.5 

Logistics 

24 

6.5.1  Evaluation  Measures 

24 

6.5.2  Measurement  Criteria 

24 

6.5.3  Analysis  Techniques  and  Tools 

24 

6.6 

Overall  Element  Analysis 

24 

7.0 

Technical  Data  (TD) 

25 

7.1 

Description 

25 

7.2 

Objective 

25 

7.3 

Logistics  Test  Measures  (LTMs) 

25 

7.4 

Reliability  and  Maintainability 

25 

7.4.1  Evaluation  Measures 

25 

7.4.2  Measurement  Criteria 

26 

7.4.3  Analysis  Techniques  and  Tools 

26 

7.5 

Human  Factors 

26 

7.5.1  Evaluation  Measures 

26 

7.5.2  Measurement  Criteria 

26 

7.5.3  Analysis  Techniques  and  Tools 

26 

7.6 

Logistics 

26 

7.6.1  Evaluation  Measures 

26 

7.6.2  Measurement  Criteria 

27 

7.6.3  Analysis  Techniques  and  Tools 

27 

7.7 

Overall  Element  Analysis 

27 

VI 


8.0  Training  and  Training  Support  (TTS)  29 

8.1  Description  29 

8.2  Objective  29 

8.3  Logistics  Test  Measures  (LTMs)  29 

8.4  Reliability  and  Maintainability  29 

8.4.1  Evaluation  Measures  29 

8.4.2  Measurement  Criteria  29 

8.4.3  Analysis  Techniques  and  Tools  30 

8.5  Human  Factors  30 

8.5.1  Evaluation  Measures  30 

8.5.2  Measurement  Criteria  30 

8.5.3  Analysis  Techniques  and  Tools  30 

8.6  Logistics  30 

8.6.1  Evaluation  Measures  30 

8.6.2  Measurement  Criteria  31 

8.6.3  Analysis  Techniques  and  Tools  31 

8.7  Overall  Element  Analysis  31 

9.0  Computer  Resources  Support  (CRS)  33 

9.1  Description  33 

9.2  Objectives  33 

9.3  Logistics  Test  Measures  (LTMs)  33 

9.4  Reliability  and  Maintainability  33 

9.4.1  Evaluation  Measures  33 

9.4.2  Measurement  Criteria  34 

9.4.3  Analysis  Techniques  and  Tools  35 

9.5  Human  Factors  35 

9.5.1  Evaluation  Measures  35 

9.5.2  Measurement  Criteria  35 

9.5.3  Analysis  Techniques  and  Tools  35 

9.6  Logistics  35 

9.6.1  Evaluation  Measures  35 

9.6.2  Measurement  Criteria  35 

9.6.3  Analysis  Techniques  and  Tools  35 

9.7  Overall  Element  Analysis  36 

10.0  Facilities  (FA)  37 

10.1  Description  37 

10.2  Objective  37 

10.3  Logistics  Test  Measures  (LTMs)  37 

10.4  Reliability  and  Maintainability  37 

10.4.1  Evaluation  Measures  37 

10.4.2  Measurement  Criteria  37 

10.4.3  Analysis  Techniques  and  Tools  37 

10.5  Human  Factors  37 

10.5.1  Evaluation  Measures  38 

10.5.2  Measurement  Criteria  38 

10.5.3  Analysis  Techniques  and  Tools  38 

10.6  Logistics  Test  38 

10.6.1  Evaluation  Measures  38 

10.6.2  Measurement  Criteria  38 

10.6.3  Analysis  Techniques  and  Tools  38 

10.7  Overall  Element  Analysis  38 

11.0  Packaging,  Handling,  Storage,  and  Transportation  (PHS&T)  39 

11.1  Description  39 

11.2  Objective  39 

11.3  Logistics  Test  Measures  (LTMs)  39 

vii 


11.4  Reliability  and  Maintainability  39 

11.4.1  Evaluation  Measures  39 

11.4.2  Measurement  Criteria  39 

11.4.3  Analysis  Techniques  and  Tools  39 

11.5  Human  Factors  39 

11.5.1  Evaluation  Measures  40 

11.5.2  Measurement  Criteria  40 

11.5.3  Analysis  Techniques  and  Tools  40 

1 1 .6  Logistics  40 

11.6.1  Evaluation  Measures  40 

11.6.2  Measurement  Criteria  40 

11.6.3  Analysis  Techniques  and  Tools  40 

1 1 .7  Overall  Element  Analysis  40 

12.0  Design  Interface  (DI)  41 

12.1  Description  41 

12.2  Objective  41 

12.3  Logistics  Test  Measures  (LTMs)  41 

12.4  Reliability  and  Maintainability  41 

12.4.1  Evaluation  Measures  42 

12.4.2  Measurement  Criteria  44 

12.4.3  Analysis  Techniques  and  Tools  44 

12.5  Human  Factors  44 

12.5.1  Evaluation  Measures  45 

12.5.2  Measurement  Criteria  45 

12.5.3  Analysis  Techniques  and  Tools  45 

12.6  Logistics  45 

12.6.1  Evaluation  Measures  45 

12.6.2  Measurement  Criteria  46 

12.6.3  Analysis  Techniques  and  Tools  46 

12.7  Overall  Element  Analysis  46 

13.0  Reporting  Procedures  47 

13.1  Introduction  47 

13.2  Combined  Test  Force  (CTF)  Reporting  47 

13.3  Logistics  Test  Squadron  Reporting  47 

13.4  Assistance  in  Writing  Technical  Reports  47 

13.5  Acknowledgements  47 

APPENDIX  A  -  References  and  Bibliography  49 

APPENDIX  B  -  Logistics  Test  Data  Sheet  (LTDS)  53 

APPENDIX  C  -  Reliability,  Maintainability,  and  Availability  (RM&A)  Parameters  and 

Methods  of  Calculation  67 

APPENDIX  D  -  List  of  Acronyms  75 

ANNEX  -  AGARD  and  RTO  Elight  Test  Instrumentation  and  Elight  Test  Techniques  A 

Series 


viii 


List  of  Figures 


Page 


Figures 

Figure  1 

Logistics  Elements 

2 

Figure  2 

Data  Analysis  Plan  Flow 

3 

Figure  3 

Logistics  Test  Measures  Numbering  System 

6 

Figure  4 

Reporting  Procedures 

47 

Figure  5 

Unit  Reporting  Procedures 

48 

List  of  Tables 

Table 

Table  1  Logistics  Test  Measures  (LTMs)  Matrix  7 


Preface 


AGARDograph  Series  160  and  300 

The  Systems  Concepts  and  Integration  (SCI)  Panel  has  a  mission  to  distribute  knowledge  concerning  advanced 
systems,  concepts,  integration,  engineering  techniques,  and  technologies  across  the  spectrum  of  platforms  and 
operating  environments  to  assure  cost-effective  mission  area  capabilities.  Integrated  defence  systems,  including 
air,  land,  sea,  and  space  systems  (manned  and  unmanned)  and  associated  weapon  and  countermeasure  integration 
are  covered.  Panel  activities  focus  on  NATO  and  national  mid-  to  long-term  system  level  operational  needs.  The 
scope  of  the  Panel  covers  a  multidisciplinary  range  of  theoretical  concepts,  design,  development,  and  evaluation 
methods  applied  to  integrated  defence  systems. 

One  of  the  technical  teams  formed  under  the  SCI  Panel  is  dedicated  to  Flight  Test  Technology.  Its  mission  is  to 
disseminate  information  through  publication  of  monographs  on  flight  test  technology  derived  from  best  practices 
which  support  the  development  of  concepts  and  systems  critical  to  maintaining  NATO’s  technological  and 
operational  superiority.  It  also  serves  as  the  focal  point  for  flight  test  subjects  and  issues  within  the  SCI  Panel 
and  ensures  continued  vitality  of  the  network  of  flight  test  experts  within  NATO. 

These  tasks  were  recognized  and  addressed  by  the  former  AGARD  organization  of  NATO  in  the  form  of  two 
AGARDograph  series.  The  team  continues  this  important  activity  by  adding  to  the  series  described  below. 

In  1968,  as  a  result  of  developments  in  the  field  of  flight  test  instrumentation,  it  was  decided  that  monographs 
should  be  published  to  document  best  practices  in  the  NATO  community.  The  monographs  in  this  series  are 
being  published  as  individually  numbered  volumes  of  the  AGARDograph  160  Flight  Test  Instrumentation  Series. 

In  1981,  it  was  further  decided  that  specialist  monographs  should  be  published  covering  aspects  of  Volume  1  and 
2  of  the  original  Flight  Test  Manual,  including  the  flight  testing  of  aircraft  systems.  The  monographs  in  this 
series  (with  the  exception  of  AG  237,  which  was  separately  numbered)  are  being  published  as  individually 
numbered  volumes  of  the  AGARDograph  300  Flight  Test  Techniques  Series. 

At  the  end  of  each  AGARDograph  160  Flight  Test  Instmmentation  Series  and  AGARDograph  300  Flight  Test 
Techniques  Series  volume  is  an  annex  listing  all  of  the  monographs  published  in  both  series. 


Abstract 


NEED 

It  has  been  understood  and  accepted  that  the  number  of  design  changes  to  new  systems  is  generally  very  high  in 
the  early  stages  of  development.  While  the  overall  costs  of  these  changes  are  low  in  the  developmental  stage, 
these  costs  quickly  escalate  as  systems  move  to  initial  fielding  and  production.  All  systems  go  through  sequential 
cost  stages.  The  stages  are:  (1)  Research,  Development,  Test  and  Evaluation  (RDT&E);  (2)  Acquisition;  and  (3) 
Operation  and  Maintenance  (O&M).  On  an  average,  60  percent  of  total  life-cycle  costs  is  consumed  on  O&M, 
which  really  means  logistics  support.  Yet,  during  test  and  evaluation,  attention  to  system  performance  factors 
and  specifications  have  been  consistently  high,  while  little  consideration  has  been  placed  on  logistics  factors  and 
specifications.  A  structured  approach  to  logistics  test  and  evaluation  is  needed  to  capitalize  on  the  correction  of 
supportability  factors  that  results  in  an  optimum  balance  between  performance  and  life-cycle  costs.  The 
approach  should  be  integrated  into  the  test  program. 


OBJECTIVE 

The  objective  of  this  AGARDograph  is  to  provide  an  introductory  overview  of  logistics  test  and  evaluation 
methods  for  supportability  testing.  This  AGARDograph  is  an  attempt  to  put  into  print  the  approach  and 
techniques  for  a  test  team  to  execute  logistics/supportability  test  and  evaluation.  To  do  so,  the 
logistics/supportability  test  and  evaluation  process  is  subdivided  into  manageable  functional  areas  and 
disciplines  called  Integrated  Logistics  Support  (ILS)  elements.  The  10  ILS  elements  are:  maintenance  planning; 
manpower  and  personnel;  support  equipment;  computer  resources;  facilities;  packaging,  handling,  storage,  and 
transportation;  and  design  interface.  Whether  a  program  is  a  large  one,  like  a  new  F-22  aircraft,  or  a  small  one, 
like  a  new  25K  Loader,  all  logistics  elements  must  be  evaluated  for  applicability  to  the  program.  The  only 
change  between  large  and  small  programs  is  the  depth  of  effort  to  be  performed  in  each  element.  Examples  will 
be  provided  to  discuss  the  test  and  evaluation  technique  to  each  area  and  are  adaptable  to  the  reader’s  particular 
area  of  interest.  This  volume  should  complement  the  AG-300  Vol.  13  on  “Reliability  and  Maintainability.” 
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1.0  INTRODUCTION 

This  data  analysis  plan  (DAP)  was  developed  by  the 
412  LSS/LGLL,  located  at  Edwards  Air  Force  Base, 
California,  United  States.  The  10  Integrated 
Logistics  Support  (ILS)  elements  defined  in  Air 
Force  Instruction  (AFI)  10-602  (Reference  1)  were 
used  as  the  baseline.  This  plan  outlines  and 
identifies  procedures  for  testing  supportability.  Since 
it  is  widely  accepted  that  60  percent  of  all  system 
costs  are  in  the  supportability  arena,  testing,  for  the 
purpose  of  reducing  costs  in  supportability,  is  in  the 
interest  of  all  NATO  countries.  Producing  the  best 
operational  availability  lAoI  at  tlie  lowest  support 
costs  is  in  the  services  and  national  interest. 

1.1  Purpose 

The  overall  purpose  of  this  DAP  is  to  describe  and 
identify  the  test  methodology,  criteria,  and  tools  for 
evaluating  and  analyzing  data  collected  for  the 
10  ILS  elements  associated  with  the  weapon  system. 
AFI  10-602  (Reference  1)  describes  and  defines  the 
10  ILS  elements  in  detail.  Figure  1  depicts  the 
relationship  between  the  1 0  ILS  elements. 

1.2  Scope 

This  DAP  is  designed  to  be  used  by  the 
developmental  test  and  evaluation  (DT&E)  and 
operational  test  and  evaluation  (OT&E) 
organizations  to  evaluate  and  analyze  the  10  ILS 
elements  supporting  the  weapon  system.  Both 
quantitative  and  qualitative  data  collected  from  the 
beginning  to  the  end  of  the  T&E  effort  will  be  used 
for  the  evaluations  and  analysis  described  in  this 
plan.  While  this  plan  does  describe  testing,  it  is  not 
intended  to  be  used  as  a  test  plan. 

1.3  System  Description 

The  overall  weapon  system  consists  of  the  air 
vehicle,  its  support  equipment,  related  facilities, 
materiel,  software,  services,  and  personnel  required 
to  ensure  the  weapon  system  can  accomplish  its 
intended  operational  role. 

1.4  Document  Organization 

This  document  is  divided  into  13  chapters  and 
5  appendices: 


Chapter  1:  Introduction  -  States  the  purpose,  scope, 
and  organization  of  the  DAP.  Chapter  2:  General 
Information  -  Contains  general  information  about 
each  of  the  participating  disciplines  addressed  within 
chapters  3-12  and  their  ILS  element  evaluation  and 
analysis  approach.  The  disciplines  addressed  in  this 
DAP  include  reliability  and  maintainability  (R&M), 
human  systems  integration  (HSI),  and  logistics. 
Chapter  2  also  provides  general  information  and 
descriptions  for  each  of  the  subsections  contained 
within  chapters  3-12  of  this  DAP.  Definitions  of  the 
logistics  test  measures  (LTMs)  referred  to  in  Table  1 
of  this  DAP  are  also  described  and  defined. 

Chapters  3  thru  12:  The  ILS  Elements  -  Describes 
and  defines  the  test  methodology,  criteria,  and  tools 
for  evaluating  and  analyzing  the  data  collected  on  the 
10  ILS  elements  of  the  weapon  system.  This  section 
is  the  main  thrust  of  the  DAP. 

Chapter  13:  Reporting  Procedures  -  Addresses  the 
reporting  procedures  for  logistics  test  and  evaluation 
(LT&E). 

Appendices:  The  appendices  to  the  plan  include: 
Appendix  A  -  References;  Appendix  B 
Questionnaires;  Appendix  C  -  Reliability, 
Maintainablity  and  Availability  (RM&A)  Parameters 
and  Methods  of  Calculation;  and  Appendix  D  -  List 
of  Acronyms. 

1.5  Test  Concept 

The  test  concept  used  for  evaluating  the  10  ILS 
elements,  as  described  in  this  DAP,  is  derived  from 
the  AFI  99-101  (Reference  2).  The  test  concept 
involves  evaluating  and  analyzing  both  the 
quantitative  and  qualitative  information  on  the  10 
ILS  elements  through  the  use  of  the  LTMs,  HSI,  and 
R&M  data. 

The  approach  consists  of  using  the  quantitative 
(R&M)  and  qualitative  (HSI  and  logistics)  measures 
written  by  these  disciplines  to  assess  the  overall 
logistics  supportability  of  the  weapon  system  or 
equipment/components.  Figure  2  depicts  this  test 
concept. 

Logistics  test  managers  should  consult  with  the  SPO 
(Chief  of  Logistics)  on  each  LTM  of  LT&E  to 
determine  specific  areas  to  be  evaluated  to  meet 
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ELEMENT  INTERRELATIONSHIPS 


Figure  1  Logistics  Elements 
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DATA  ANALYSIS  PLAN  FLOW 


METHODS  OF  EVALUATION  BY 
LOGISTICS  TEST  MEASURE 


Figure  2  Data  Analysis  Plan  Flow 
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customer  requirements  and  needs.  For  example,  in 
the  ILS  area  of  support  equipment,  does  the 
SPO/COL  want  LTM  34  (safety)  and  LTM  35 
(functionality)  evaluated?  It  could  be  either  or  both. 


Technical  reports  (TR)  may  be  written  by  2-digit 
work  unit  code  (WUC),  logistics  control  number 
(LCN),  or  system,  subsystem,  sub-subsystem  number 
(SSSN)  structure  and  organized  by  logistics  elements 
(i.e.,  airframe/ structure,  engines,  electrical)  and 
grouping  is  permissable  in  areas  like 
communications  where  radios,  interphone,  and 
SATCOM  can  be  evaluated  together.  Grouping  is 
done  by  similarity  and  driven  by  the  quantity  of  data. 
Technical  reports  may  by  grouped  and  titled  as 
following: 


Support  General 
Storage/Delivery 
Oxygen 

Communications 
Landing  Gear 
Avionics 
Electrical 
Gun  Systems 
Integrated  Diagnostics 


Weapons 

Built-in-Test 

Propulsion 

APU/JFS 

Flight  Control  Systems 
Gun  Systems 
Environmental  Systems 
Fuel 


Deficiencies  discovered  during  testing  will  be 
identified  through  one  or  more  of  the  following 
methods:  watch  item  tracking  (WIT),  deficiency 
reports  (DRs),  and  publication  change  requests 
(PCRs). 

Watch  item  listings  are  for  identification  of 
suspected  or  potential  hardware  and  software 
deficiencies.  Watch  item  tracking  is  used  for  early 
identification  of  potential  DRs. 

Deficiency  reports  will  be  initiated  to  identify  known 
hardware  or  software  deficiencies  or  recommended 
enhancements  in  various  systems,  subsystems,  or 
support  equipment. 

Publication  change  requests  will  be  initiated  and 
controlled  through  the  technical  order  validation  and 
verification  activity. 
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2.0  GENERAL  INFORMATION  ABOUT  THE  STRUCTURE  AND  CONTENTS  OF 
CHAPTERS  3  -  12,  THE  ILS  ELEMENTS 


2.1  Introduction 

This  section  provides  general  information  about  the 
structure  and  contents  of  chapters  3-12,  the  ILS 
elements.  Specifically,  it  provides  general 
information  about  each  of  the  disciplines  and  how 
they  relate  to  each  of  the  ILS  elements.  The 
disciplines  addressed  include  R&M,  HSI,  and 
logistics.  This  section  also  provides  general 
information  and  descriptions  of  the  subheadings 
addressed  in  chapters  3-12.  Descriptions  and 
definitions  of  the  LTMs,  as  referred  to  in  chapters 
3-12  of  this  DAP,  are  also  provided. 

Chapters  3-12  of  this  DAP  have  the  same  layout  and 
structure  shown  below  for  each  of  the  1 0  ILS 
elements  discussed. 

Description.  Provides  a  detailed  description  of  the 
ILS  element.  It  also  identifies  the  scope  of  the 
evaluations  and  analysis  to  be  conducted  under  the 
DAP  efforts. 

Objective.  Describes  the  overall  objectives  to  be 
reached  for  the  specific  ILS  element  evaluated  and 
analyzed.  These  objectives  support  the  test  methods 
referenced  in  Evaluation  Measures  section. 

Logistic  Test  Measures  (LTMs).  The  LTMs  are  the 
smallest  unit  of  measurement  on  which  the  DAP  is 
built.  Each  LTM  is  defined  to  partition  each  of  the 
ILS  elements  into  manageable  and  useful  units  of 
measure.  The  LTMs  provide  a  method  of  measuring 
specific  performance  and  supportability  parameters 
for  each  of  the  10  ILS  elements.  The  LTMs  support 
the  evaluation  measures  referenced  in  the  Evaluation 
Measures  section,  using  the  specified  objectives 
defined  in  the  Objectives  section  as  a  baseline. 
Identifies  the  LTMs  used  by  each  discipline, 
logistics,  R«&M,  and  HSI  to  perform  the  evaluation 
and  analysis  of  the  ILS  elements.  The  LTMs 
identified  in  the  ILS  element  chapters  correlate  to  the 
LTM  numbering  system  (Figure  3),  LTM  matrix 
(Table  1),  and  LTM  definitions  located  in  chapters 
3-12. 

Reliability  and  Maintainability.  The  R&M 

evaluations  and  analyses,  as  addressed  in  this  DAP, 
will  identify  the  measured  R&M  and  highlight 
critical  areas  as  the  detail  design  evolves  and 


matures  through  the  incorporation  of  changes  to 
improve  R&M. 

Evaluation  Measures.  Identifies  the  test  measures 
for  evaluating  the  specific  performance  and 
supportability  aspects  of  the  10  ILS  elements. 

Measurement  Criteria.  Establishes  and  identifies 
the  success  criteria  for  evaluating  the  respective 
LTM.  The  identified  criteria  establishes  the 
constraints  and  critical  measures  to  be  applied  in  the 
evaluation  and  analysis  of  the  10  ILS  elements  using 
the  LTMs  as  a  baseline. 

Analysis  Techniques  and  Tools.  The  logistic 
support  analysis  report  (LSAR)  database  will  be 
compared  to  the  ILS  analysis  and  evaluation 
activities  conducted  under  this  DAP.  The  LSAR 
database  information,  as  defined  in  the  contract, 
together  with  the  RM&A  data  will  be  used  to  provide 
a  comprehensive  picture  of  the  supportability  aspects 
of  the  system.  The  LSAR  database  provides  and 
identifies,  both  at  the  system  and  subsystem  level, 
many  quantitative  RM&A  and  ILS  requirements 
useful  for  the  evaluations. 

Human  Factors  (HSI).  Human  Factors  is 
concerned  with  how  people  receive  information 
through  use  of  their  senses,  store  this  information, 
and  process  it  in  making  decisions. 

Logistics  Test.  Logistics  (maintenance)  efforts  focus 
on  assessing  system  supportability  through  direct 
man/machine  interface.  Logistics  evaluates  the  10 
ILS  elements  from  both  a  qualitative  and  quantitative 
perspective. 

Evaluation  Measures.  Do  you  want  information  in 
here? 

Measurement  Criteria.  Ditto  from  above. 

Analysis  Techniques  and  Tools.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  to  determine 
their  validity.  All  comments  of  a  test-unique  nature 
will  be  excluded  from  analysis.  All  comments 
applicable  to  production-representative  aircraft  or 
systems  will  be  analyzed  to  determine  the  root  cause 
of  the  deficiency.  Data,  when  analyzed, 


6 


LOGISTICS  TEST  MEASURE  (LTM) 

NUMBERING  SYSTEM 


ELEMENT 

LTM 

MAINTENANCE  PLANNING 

10-19 

MANPOWER  &  PERSONNEL 

20-29 

SUPPORT  EOUIPMENT 

30-39 

SUPPLY  SUPPORT 

40-49 

TECHNICAL  DATA 

50-59 

TRAINING 

60-69 

COMPUTER  RESOURCES 

70-79 

FACILITIES 

80-89 

PHS&T 

90-99 

DESIGN  INTERFACE 

100-109 

Figure  3  Logistics  Test  Measures  Numbering  System 


7 


Table  1 

LOGISTICS  TEST  MEASURES  (LTMs)  MATRIX 


ELEMENTS 

LOGISTICS  TEST 

R&M 

HUMAN  FACTORS 

Chapter  3 

Maintenance  Planning 

Scope 

Frequency 

Task  Times 

Reliability 

Maintainability 

N/A 

Chapter  4 

Manpower  and  Personnel 

Crew  Size  Adequacy 

AFSC  and  Skill  Level 

Reliability 
Maintainability 
Average  Crew  Size 
AFSC 

Human  Performance 

Chapter  5 

Support  Equipment 

Functionality 

Durability 

Setup/Disassembly 

Reliability 

Maintainability 

Ease  of  Use 
Handling 

Safety 

Chapter  6 

Supply  Support 

Source,  Maintenance,  and 
Recoverability  (SMR)  Codes 
Availability 

Reliability 

N/A 

Chapter  7 
Technical  Data 

Task  Requirements 
Reference  Material 

Reliability 

Maintainability 

Safety 

Effectiveness  of 
Instructions 

Chapter  8 

Training  and  Training 
Support 

Knowledge  Training  - 
Type  1&4 

Proficiency  Training 

Reliability 

Maintainability 

Safety 

Chapter  9 

Computer  Resources 
Support 

Functional  Utility 
Resource  Effectiveness 

CRS  Reliability 

CRS  Maintainability 
Diagnostics  System 
Adequacy 

Ease  of  Use 

Chapter  10 
Facilities 

Utilities 

Capacity 

Reliability 

Maintainability 

Safety 

Chapter  1 1 

Packaging,  Handling, 
Storage,  and 

Transportation  (PHS&T) 

Suitability 

Reliability 

Safety 

Chapter  12 

Design  Interface 

Machine/Machine  Interface 
Design 

Interoperability 

Reliability 

Maintainability 

Human/Machine 

Interface 

Safety 
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can  be  grouped  by  systems  (i.e.,  electrical,  ECS, 
aircraft  power  plants,  avionics,  airframe,  etc.).  The 
LTMS  database  will  be  used  as  a  source  of 
information  for  evaluation. 

Overall  Element  Analysis.  Describes  how  results 
from  all  three  disciplines  are  combined  to  form  a 
single-rating  and  conclusion  for  the  element. 
Identification  of  impacts  to  the  specific  element,  as 
well  as  impacts  to  the  other  ILS  elements  will  be 
described. 

2.2  General  Information  about  the 
Disciplines  Referenced  in  Chapters 
3-12,  The  ILS  Elements 

2.2.1  Logistics  Test 

Logistics  (maintenance)  efforts  focus  on  assessing 
system  supportability  through  direct  man/machine 
interface.  Logistics  evaluates  the  10  ILS  elements 
from  both  a  qualitative  and  quantitative  perspective. 
The  specific  qualitative  and  quantitative  measures 
used  are  listed  in  the  appendix  B.  Maintenance 
personnel  orchestrate  the  day-to-day  ILS  element 
activities  and  collect  data  necessary  for  evaluating 
the  air  vehicle  and  system  specifications. 

2.2. 1.1  System  Specifications  and 
Logistics  Test 

The  following  system  specification  paragraphs  were 
used  for  guidance  in  developing  the  LT&E 
questionnaire  and  consequently  the  Logistics  Test 
sections  of  this  DAP  for  the  system  under  test. 
Similar  specifications  for  your  system  must  be 
reviewed  for  developing  logistics  test  sections  of 
your  DAP. 

1.  Losistics.  The  air  vehicle  and  support 
equipment  shall  be  designed  with  a  primary  goal  of 
minimizing  the  life-cycle  costs  and  to  enhance  the 
accomplishment  of  required  maintenance.  Items  for 
which  a  recurring  maintenance  requirement  is 
expected  shall  be  so  located  and  configured  as  to 
enhance  fault  isolation,  servicing,  removal  and 
replacement  without  compromising  rapid  turnaround 
capability.  Support  equipment  shall  be  configured  to 
be  compatible  with  the  deployability  requirements  of 
the  air  vehicle. 

2.  Maintenance.  The  system,  exclusive  of 
aircraft  engines,  shall  be  compatible  with  two  or 
three  levels  of  maintenance.  The  organizational 
level  of  maintenance  shall  require  a  minimum  of  skill 
levels,  personnel,  and  facilities.  The  System  shall  be 
provided  with  the  capability  of  performing 
100-percent  fault  isolation,  to  the  line  replaceable 
unit  (LRU)  level,  at  the  organizational  level  using 


support  equipment  (SE),  built-in-test  (BIT), 
Technical  Orders  (T.O.s),  or  any  combination  of  the 
three.  The  design  shall  allow  access  at  the  base  level 
to  areas  that  are  normally  inspected  during  a  depot 
inspection. 

3.  Maintenance  Manning.  The  system  shall  be 
designed  such  that  it  can  be  operationally 
maintained  primarily  by  3-  and  5-level  personnel 
using  fully  proceduralized  job  performance  aid 
materials  developed  in  accordance  with 
system  specifications. 

4.  Preventive  Maintenance.  The  system  shall 
employ  a  reliability  centered  maintenance  (RCM). 
The  system  shall  be  designed  such  that  scheduled 
inspection  requirements  shall  be  the  minimum 
required  to  prevent  degradation  of  equipment  safety 
and  capability  levels.  The  system  shall  enable  the 
maintenance  preflight  inspection  to  be  accomplished 
by  one  5-level  mechanic  in  less  than  2  hours, 
thruflight  inspection  in  less  than  1  hour,  and  the  look 
phase  of  the  postflight  in  less  than  2  hours.  No 
special  tools  shall  be  required  for  preflight, 
thruflight,  or  postflight  inspections. 

5.  Supply.  The  design  requirements  shall 
favor  maximum  use  of  standard  (common  use)  parts, 
SE  accessories,  and  components  unless  single 
purpose/peculiar  items  are  shown  to  be  more  cost 
effective.  Transportation  modes,  policies,  and 
procedures  already  in  existence  during  the  life  cycle 
of  the  system  will  be  used  to  support  the  system. 
Packaging  for  supply  or  resupply  will  conform  to  the 
principles  of  the  most  economical  overall  cost  and 
necessary  protection  to  prevent  damage  or 
deterioration  during  shipment  and  storage. 
Handling  requirements  will  be  compatible  with  the 
best  commercial  practices. 

6.  Facilities  and  Facilities  Equipment.  The 
design  of  the  aircraft  and  aircraft  support  equipment 
shall  consider  maximum  utilization  of  existing  Air 
Force  facilities.  These  facilities  include,  but  are  not 
limited  to,  alert  facilities,  training  facilities, 
operations  buildings,  maintenance  hangars,  docks, 
shops,  and  test  cells. 

7.  Training.  The  training  of  personnel  shall 
be  identified  by  the  Instructional  Systems 
Development  (ISD)  process  and  will  be  conducted  by 
the  Air  Force.  Contractor  training  (Type  1)  shall  be 
required  for  ISD,  DT&E,  and  lOT&E  personnel  to 
safely  operate,  maintain,  and  evaluate  the  system. 

8.  Support  Equipment.  Support  equipment 
functional  characteristics,  in  addition  to  those 
specified  herein,  shall  be  specified  in  the  SE  General 
specification  and  SE  item  specification.  The  SE  shall 
provide  the  operational  support  capability  necessary 
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for  the  weapon  system  to  meet  its  performance, 
availability,  alert,  turnaround,  and  maintainability 
requirements  specified  herein.  This  support 
capability  shall  be  provided  within  the  constraints  of 
the  deployment  and  maintenance  concepts  specified 
herein  and  A  ir  Force  personnel  capabilities. 

2.2. 1.2  Logistics  Developmental  Test  and 
Evaluation  (DT&E)  of  Data 

Logistics  data  will  be  collected  through  the  use  of 
logistics  test  data  sheets  (LTDS)  questionnaires  that 
are  completed  by  maintenance  personnel  who 
perform  maintenance  tasks  on  the  test  article.  The 
LTDSs  will  be  completed  for  all  specific  test 
information  sheet  (TIS)  test  points  identified  in  the 
integrated  test  plan. 

2.2.2  Reliability  and  Maintainability 

The  R&M  evaluations  and  analyses,  as  addressed  in 
this  DAP,  will  identify  the  measured  R&M  and 
highlight  critical  areas  as  the  detail  design  evolves 
and  matures  through  the  incorporation  of  changes  to 
improve  R&M.  The  R&M  test  methods,  as 
discussed  within  this  DAP,  are  the  test  tools  used  to 
evaluate  each  R&M  aspect  of  the  10  ILS  elements 
and  their  ability  to  meet  their  specified  reliability, 
maintainability,  and  performance  levels  of 
achievement. 

2.2.2. 1  System  RM«&A  Requirements 

All  the  quantitative  system  RM&A  requirements  are 
contained  and  identified  in  the  reliability  and 
maintainability  allocations,  assessment,  and  analysis 
reports.  These  reports  contain  comprehensive 
summaries  of  the  system  and  subsystem  level  R&M 
and  ILS  parameters. 

2.2.3  Human  Factors  (HSI) 

Human  Factors  is  concerned  with  how  people 
receive  information  through  use  of  their  senses,  store 
this  information,  and  process  it  in  making  decisions. 
The  primary  focus  of  Human  Factors  is  the 
evaluation  of  human/machine  interaction  during  the 
performance  of  a  task.  The  science  of  Human 
Factors  is  also  called  ergonomics.  Human  Factors  is 
also  concerned  with  the  design  of  structures, 
communication  systems,  safety,  and  learning 
processes.  Since  the  user  of  machines  is  man,  human 
characteristics  must  be  considered  in  their  design  and 
construction.  The  task  which  confronts  Human 
Factors  is  to  describe  the  special  abilities  and 
limitations  of  humans  in  such  a  way  that  design 
engineers  can  effectively  incorporate  the  human 


operator  as  a  component  in  the  human/machine 
system. 

2.2.3. 1  Weapon  System  Human  Factors 
Requirements 

All  weapon  system  Human  Factors  requirements  are 
contained  in  the  test  article  specification.  The 
following  is  an  example  of  a  HSI  air  vehicle 
specification  (AVS).  The  following  examples  are 
AVS  paragraphs  which  applies  to  Human  Factors 
and  illustrates  how  Human  Factors  may  be  specified 
for  a  system: 

1.  Paragraph  3.3.7  Human  Factors  Engineering 
(HFEI.  Unless  otherwise  contractually  specified,  the 
general  HFE  requirements  in  M1L-STD-1472B 
(Reference  3),  Section  4,  and  specific  AVS 
paragraphs  shall  apply. 

2.  Paragraph  3.3.7. 1  Crew  svstem/workload 
integration.  Crew  workspaces  shall  be  designed  for 
maximum  utility  and  efficient  use  by  user  personnel 
in  the  performance  of  system  tasks.  Displays  shall 
provide  the  operator  with  clear  indications  of 
equipment  or  system  conditions.  The  indications 
shall  be  within  the  perceptual  capabilities  of  the 
operators.  Controls  shall  provide  the  operators  with 
means  to  control  equipment  or  system  conditions  and 
shall  be  within  the  physiological  and  mental 
capabilities  of  the  operators.  Controls  shall  be 
compatible  with  their  associated  displays. 

3.  Paragraph  3.3.7.11  Human  factors  design 
for  maintainability.  The  air  vehicle  shall  be  designed 
for  maintainability  as  specified  in  MIL-STD-1472B, 
(Reference  3)  paragraphs  5.9.1  through  5.9.11.2  and 
5.9.11.5  through  5.9.18. 

2.2.3.2  Human  Factors  Developmental 
Test  and  Evaluation  (DT&E)  Data 

Human  Factors  data  will  be  collected  through  the  use 
of  LTDS  questionnaires  distributed  to  maintenance 
personnel  who  perform  maintenance  on  the  test 
article.  Logistics  test  data  sheets  will  be  completed 
for  all  test  information  sheets  (TIS)  test  points 
identified  in  the  flight  test  plan  (FTP)  or  test  and 
evaluation  master  plan  (TEMP).  The  LTDSs  will  be 
reviewed  by  Human  Factors  and  or  Logistics  test 
evaluators  for  completeness  and  accuracy.  Data 
from  the  questionnaire  will  be  loaded  into  the  LTMS 
database  for  use  in  analysis  of  each  ILS  element. 
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3.0  MAINTENANCE  PLANNING  (MP) 

3.1  Description 

Maintenance  planning  (MP)  includes  all  planning 
and  analysis  associated  with  the  establishment  of 
requirements  for  the  overall  support  of  the  aircraft 
system  throughout  its  life  cycle.  Maintenance 
planning  describes  the  evolving  maintenance 
concepts,  plans,  and  requirements  for  on-  and  off- 
equipment  maintenance  for  the  system.  Maintenance 
planning  relies  heavily  on  the  reliability  centered 
maintenance  (RCM)  principle  that  establishes 
maintenance  tasks  based  on  design  reliability  and 
logistic  support  analysis  (LSA).  These  concepts  and 
plans  include  the  levels  of  maintenance,  maintenance 
environment,  limitations,  constraints,  requirements, 
failure  diagnostic  techniques,  depot  maintenance 
support,  contractor  support,  and  contractor 
warranties.  The  concepts  and  plans  also  address  the 
extent  and  usage  of  support  methods  such  as  spares, 
T.O.s,  facilities,  SE,  training,  and  other  ILS 
elements.  An  evaluation  of  the  impact  of  the  other 
nine  ILS  elements  on  MP  will  be  performed  as 
outlined  in  section  3.6  of  this  element. 

3.2  Objective 

The  purpose  of  the  MP  evaluation  is  to  determine  the 
effectiveness  of  the  maintenance  concept  at  the 
organizational  and  depot  level  to  support  the  system. 
The  evaluation  will  ensure  that  preventive  MP  is 
consistent  with  the  RCM  concept;  support  efforts  and 
maintenance  intervals  are  minimized;  planned 
maintenance  tasks  are  optimally  structured;  and 
preventive  maintenance  task  times  are  standardized. 

3.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  MP 
element: 

Reliability  &  Maintainability 

LTM  10  -  Reliability 
LTM  1 1  -  Maintainability 
Human  Factors  -  None 

Logistics  Test 

LTM  12 -Scope 

LTM  1 3  -  Frequency 
LTM  14  -  Task  Times 


3.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  MP  element  and  are  addressed  in  detail 
under  section  3.4.1. 

LTM  10  -  Reliability  (Hardware) 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

Mean  Time  Between  Removal  —  MTBR 

LTM  11  -  Maintainability 

Maintenance  Man-Hours  per  Flight  Hour  — 

MMH/FH 

Mean  Man-Hours  to  Repair  —  MMTR 

3.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  the  LTMs  cited  in  section  3.3  to  the  MP 
logistic  element  of  the  aircraft  system: 

LTM  10  -  Reliability 

The  aircraft  system  reliability  will  be  evaluated 
under  this  LTM.  This  LTM  evaluates  the  optimal 
reliability  performance  characteristics  of  the  aircraft 
system.  The  objective  is  to  determine  if  the  planned 
reliability  and  ILS  criteria  for  the  MP  element  have 
been  met. 

Mean  Time  Between  Maintenance  (Inherent). 
MTBM(I)  measures  the  mean  time  between 
unscheduled  on-equipment  maintenance  actions 
caused  by  design  or  manufacturing  defects  with  time 
expressed  in  aircraft  flying  hours.  This  measure 
includes: 

•  chargeable  inherent  maintenance  actions 

•  unscheduled  maintenance 

•  on-equipment  maintenance  (line  or 
organizational  level) 

•  time  in  flying  hours. 

MTBM(I)  excludes  test-unique  actions.  Test-unique 
actions  are  those  actions  that  support  engineering 
evaluations,  modifications,  etc.  Maintenance  actions 
that  are  in  direct  support  of  engineering  test  and 
evaluation  of  system  or  subsystem  or  component 
modifications,  TO  validation,  or  SE  compatibility 
will  be  considered  nonrelevant  to  both  the 
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contractual  and  optional  R&M  evaluation.  The 
following  formula  describes  MTBM(I): 

Formula  1 : 


JUTALFUGHTHOURS 

MiBm)  = - 

TOTALn^mREMhMNmmKEAcncm 


where: 

MTBM(I)  is  the  mean  time  between  inherent 
maintenance  actions.  Total  inherent  maintenance 
actions  are  those  failures  which  result  from  internal 
cause.  These  failures  are  the  result  of  defective 
design  or  manufacture  and  will  be  coded  with  a  Type 
1  how  malfunction  code  (HMC)  and  the  appropriate 
action  taken  code  (ATK).  If  a  removed  component 
is  later  found  to  be  serviceable,  the  action  will  be 
changed  to  a  NO-DEFECT  maintenance  action. 
There  will  be  a  maximum  of  one  chargeable 
maintenance  action  with  an  ATK  G  and  Type  1 
HMC  combination  for  a  given  WUC  within  a  given 
job  control  number  (JCN).  Total  inherent 
maintenance  actions  will  be  extracted  from  the 
system  effectiveness  data  system  (SEDS)  failure 
summary  report.  Total  flying  hours  will  be  extracted 
from  the  combined  test  force  (CTF)  debrief  database 
and/or  the  aircraft  R&M  cyclic  database. 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  measures  the  weapon 
system  reliability.  MTBM(C)  is  mean  time  between 
unscheduled  on-equipment  corrective  maintenance 
actions  with  time  expressed  as  aircraft  flying  hours. 
Where  total  corrective  maintenance  actions  is  the 
sum  total  of  the  number  of  Type  1  (inherent),  Type  2 
(induced)  and  Type  6  (NO  DEFECT,  Cannot 
Duplicate  [CND],  or  to  facilitate  other  maintenance 
[FOM])  actions.  Detailed  descriptions  and 
definitions  of  these  types  of  maintenance  actions  are 
found  under  the  SEDS  Data  Collection  Procedure 
Guide.  This  measure  includes: 

•  corrective  maintenance  actions  (sum  of 
inherent,  induced,  and  no  defect) 

•  unscheduled  maintenance 

•  time  in  flying  hours  (production  aircraft) 

•  on-equipment  maintenance. 

Formula  2: 


TOTAL  FucnrHjm 

MBMq= - 

TOTAL  amECnVE  MmTTNANOE  ACHCNS 


where: 

MTBM(C)  is  the  mean  time  between  maintenance 
with  total  corrective  actions.  Total  flying  hours  can 


be  obtained  from  the  CTF  debrief  database  and/or 
the  aircraft  R&M  cyclic  database.  Total  corrective 
maintenance  actions  are  derived  from  SEDS  failure 
reports,  which  include  the  sum  of  the  inherent 
induced  and  no  defect  maintenance  actions. 

Mean  Time  Between  Removal  (MTBR).  MTBR 
measures  the  mean  time  interval  between  removal  of 
repairable  components,  with  time  expressed  as 
aircraft  flying  hours.  MTBR  includes: 

•  production  aircraft  flying  hours 

•  indicated  failures  (retest  OK  [RTOK]) 

•  chargeable  removals: 

-  on  equipment 

-  repairable. 

MTBR  excludes: 

•  removals  to  facilitate  other  maintenance 

•  Time  Complaince  Technical  Orders 
(TCTOs) 

•  scheduled  maintenance 

•  nonrepairable  components. 

Formula  3: 


_  TOTAL  FUGHT  HOURS 

MIBR^ - 

TOTAL  NUMBER  CF  REMOVALS 


where: 

MTBR  is  the  mean  time  between  removals.  Total 
flight  hours  can  be  obtained  from  the  CTF  debrief 
database  and\or  the  aircraft  R&M  cyclic  database. 
Total  removals  or  retrieval  maintenance  actions  can 
be  derived  from  work  field  <1>  of  SEDS  failure 
reports. 

LTM  11  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour 
(DMMH/FH).  DMMH/FH  measures  the 
maintenance  hours  per  flight  hour  required  to 
accomplish  the  maintenance  activities  directly 
related  to  the  maintenance  of  the  air  vehicle  system. 
DMMH/FH  includes: 

•  Maintenance  man-hours  that  are  controllable 
through  design,  such  as: 

-  repair  times 

-  detection  capability 

-  isolation  times 

-  CND  and  RTOK  rates 

-  failure  frequency 

-  number  of  personnel  required  to  perform  a  task 

-  T.O.s  and  training. 
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•  Maintenance  man-hours  for  both  scheduled  and 
unscheduled  maintenance  attributed  to 
corrective  and  or  preventive  actions  and 
inspections. 

•  Time  in  flying  hours. 

DMMH/FH  excludes: 

•  Maintenance  man-hours  to  accomplish  preflight, 
postflight,  and  thruflight  inspections. 

Formula: 


TOTAL  DIRECT  MAINTENANCE  MAN-HOURS 

DMMH / FH  = - 

TOTAL  FLYING  HOURS 


Mean  Time  to  Repair  (MTTR).  MTTR  measures 
the  mean  time  required  to  complete  a  maintenance 
action.  MTTR  includes: 

•  Maintenance  time  that  is  controllable 
through  design,  such  as: 

-  repair  times 

-  detection  capability 

-  isolation  times 

-  CND  and  RTOK  rates 

-  failure  frequency 

-  number  of  personnel  required  to  perform  a 
task 

-  T.O.s  and  training 

•  Maintenance  man-hours  from  both  schedule 
and  unscheduled  maintenance  attributed  to 
corrective  and  or  preventive  actions  and 
inspections. 

MTTR  excludes: 

•  Maintenance  man-hours  to  accomplish 
preflight,  postflight,  and  thruflight 
inspections. 

Formula : 


.  TOTAL  MAINTENANCE  TIME 

MTTR — - nm - 

TOTAL  MAINTENANCE  ACTIONS 


The  total  maintenance  time  (clock  hours)  and  the 
total  maintenance  actions  will  be  computed  from 
maintenance  data  recorded  in  the  SEDS  or  core 
automated  maintenance  system  (CAMS)  database. 

3.4.2  Measurement  Criteria 

The  main  objective  of  this  ILS  element,  as  it  pertains 
to  R&M,  is  to  verify  if  the  planned  aircraft  R&M 
allocations  are  met.  The  evaluation  and  analysis  will 


identify  any  R&M  trends/shortfalls  which 
significantly  impact  supportability  or  the  ability  of 
the  system  to  meet  the  specified  R&M  allocations. 
These  objectives  will  be  met  by  verifying  the 
attainment  of  the  technical  R&M 
requirements/ specifications,  evaluation  of  the 
logistics  questionnaires  (LTDSs),  DRs,  and  PCRs 
effecting  the  maintenance  plan. 

3.4.3  Analysis  Techniques  and  Tools 

Applications  of  SEDS  and  LSAR  databases  will 
assist  in  identifying  R&M  design-related  shortfalls. 
Actual  flight  test  values  obtained  from  SEDS  will  be 
compared  to  predicted  values  to  determine 
systems/components  to  be  targeted  for  investigation 
using  LCC  analysis.  Reliability  and  maintainability 
growth  analysis  programs,  Weibull  analysis,  Pareto 
charts,  SEDS  listings  and  failure  analysis  data  from 
FRACAS  will  be  used  to  analyze  maintenance  data. 
When  R&M  shortfalls  pertaining  to  maintenance 
planning  are  discovered,  DRs  will  be  generated  and 
forwarded  through  the  system  program  office  (SPO), 
appropriate  integrated  product  team  (IPT),  or  system 
program  manager  (SPM). 

3.5  Human  Factors 

The  Air  Force  Flight  Test  Center  (AFFTC)  mission 
for  Human  Factors  test  and  evaluation  is  to  test 
system  designs  for  compatible  interaction  among 
human,  machine,  and  environment,  assuring 
effeetive  system  operations  by  assigned  Air  Force 
personnel.  This  element  examines  planning  and, 
therefore,  is  not  an  issue  for  Human  Factors 
engineering. 

3.6  Logistics  Test 

The  MP  evaluation  will  focus  on  the  results  of 
demonstrated  maintenance  actions  and  the  impact 
those  actions  have  on  the  maintenance  concepts  for 
the  aircraft  and  related  systems.  The  scope  of 
scheduled  inspections  and  maintenance  actions, 
frequency  of  scheduled  and  unscheduled  tasks,  and 
the  impact  of  task  completion  times  will  be 
considered  during  this  evaluation. 

The  following  LTMs  influence  the  MP  element  and 
are  addressed  in  detail  in  section  3.6.1: 

LTM  12 -Scope 

LTM  13  -  Frequency 

LTM  14  -  Task  Completion  Times 

3.6.1  Evaluation  Measures 

Logistics  test  measures  12,  13,  and  14  will  be 
evaluated  to  determine  the  effectiveness  of  scheduled 
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and  unscheduled  maintenance  at  the  organizational 
level  to  support  the  system. 

LTM  12  -  Scope 

Scope  refers  to  the  extent  of  coverage.  This  LTM  is 
designed  to  evaluate  the  adequacy  of  scheduled  and 
unscheduled  maintenance  actions  in  relation  to  task 
coverage  and  to  identify  requirements  that  should  be 
added  to  or  deleted  from  the  particular  tasks. 
Attention  should  be  focused  on  whether  or  not 
coverage  of  the  items  was  thorough  enough  without 
excessive  requirements.  Rate  the  adequacy  of  task 
coverage  in  relation  to  minimal/excessive 
requirements. 

LTM  13  -  Frequency 

Frequency  refers  to  the  rate  of  occurrence.  This 
LTM  is  designed  to  evaluate  the  adequacy  of 
scheduled  maintenance  intervals  on  the  particular 
item/ system  and  to  identify  deficient  areas  requiring 
frequent  unscheduled  maintenance.  By  evaluating 
the  frequency  of  scheduled  maintenance  intervals, 
we  can  determine  whether  the  intervals  are  being 
optimized  to  prevent  premature  degradation  to 
aircraft  components  and/or  to  prevent  unnecessary 
aircraft  downtime.  Rate  the  frequency  of  the 
inspection/task  performed  for  adequacy  considering 
whether  intervals  are  too  long  or  too  short. 

LTM  14  -  Task  Completion  Time 

Task  completion  time  refers  to  the  time  required  to 
perform  scheduled  or  unscheduled  maintenance 
tasks.  This  LTM  is  designed  to  evaluate  the 
adequacy  of  task  time  considering  whether  or  not  the 
completion  time  was  reasonable.  Tasks  requiring 
excessive  time  to  accomplish  can  negatively  impact 
MP  and  will  be  identified  under  this  LTM.  Rate  the 
adequacy  of  task  completion  time  considering  all 
unexpected  circumstances  that  increased  task 
completion  time. 

3.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDSs  and  there  are  no 
significant  negative  comments  noted.  Logstics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools.  When  evaluated  with  the  R&M  LTMs,  MP 
can  be  properly  assessed. 


3.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

3.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single  rating  conclusion  for  the 
ILS  element  based  on  mission  impact. 

In  order  to  make  an  accurate  assessment  of  the  MP 
element,  the  impact  of  all  other  ILS  elements  has  to 
be  taken  into  consideration.  Maintenance  Planning 
will  be  evaluated  for  each  2-  and  3 -digit  WUC  for 
each  of  the  three  disciplines  (LT,  R&M,  and  HF).  In 
addition,  data  from  the  following  elements  will  be 
combined  to  evaluate  the  impact  of  integrated  ILS 
elements  on  MP; 

1 .  Manpower  &  Personnel  (M&P) 

2.  Support  Equipment  (SE) 

3.  Supply  Support  (SS) 

4.  Technical  Data  (TD) 

5.  Training  and  Training  Support  (TTS) 

6.  Computer  Resources  Support  (CRS) 

7.  Facilities  (FA) 

8.  Packaging,  Handling,  Storage,  and 
Transportation  (PHS&T) 

9.  Design  Interface  (DI) 

Data  that  indicates  problems  with  a  WUC  for  any  of 
the  three  disciplines  will  be  evaluated  across  all  the 
ILS  elements  from  that  WUC. 

For  Example:  R&M  data  may  indicate  a  high 
MTTR  for  a  specific  WUC.  Data  for  that  WUC  will 
be  examined  for  all  10  ILS  elements  in  all 
disciplines.  If  for  example,  logistics  test  data 
indicates  training  was  less  than  adequate  and  HF 
data  indicates  SE  was  difficult  to  utilize,  the 
combined  impact  of  these  elements  across  the  three 
disciplines  identify  the  problem  which  must  be 
corrected  to  reduce  the  MTTR  and  positively 
influence  R,  M,  and  A  for  the  task. 
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4.0  MANPOWER  AND  PERSONNEL  (M&P) 


4.1  Description 

Manpower  and  personnel  (M&P)  pertains  to  the 
identification  of  military  and  civilian  personnel 
requirements  by  Air  Force  Specialty  Codes  (AFSCs) 
and  skill  levels  necessary  to  maintain/support  the 
aircraft  weapons  system.  Manpower  and  personnel 
requirements  are  identified  through  the  LSA  process 
for  all  tasks  required  to  support  the  aircraft  and 
related  systems.  Data  will  be  collected  and  analyzed 
to  evaluate  M&P  requirements  during  flight  testing. 
These  evaluations  will  assist  in  the  establishment  of 
manning  for  future  aircraft  maintenance  operations. 

4.2  Objective 

To  determine  the  quantity  of  personnel  and  skill 
levels  required  to  accomplish  maintenance/support 
tasks  for  the  aircraft  and  related  systems. 

4.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  M&P 
element: 

Reliability  and  Maintainability 

LTM  20  -  Reliability 

LTM  21  -  Maintainability 

LTM  22  -  Average  Crew  Size 

LTM  23  -  Air  Force  Specialty  Code  (AFSC) 

Human  Factors 

LTM  24  -  Human  Performance 

Logistics 

LTM  25  -  Crew  Size  Adequacy 

LTM  26  -  Air  Force  Specialty  Code  (AFSC)  and 

Skill  Level 

4.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  M&P  element  and  are  addressed  in  section 
4.4.1: 

LTM  20  -  Reliability 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

LTM  21  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour  — 
DMMH/FH 


LTM  22  -  Average  Crew  Size 
Maintenance  Man-Hours  (MMH) 

Elapsed  Times  (ET) 

Average  Crew  Size 

LTM  23  Air  Force  Specialty  Code  (AFSC) 

4.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  the  LTMs  cited  in  section  4.3  to  M&P. 

LTM  20  -  Reliability 

Increased  aircraft  system  reliability  has  a  direct 
payoff  visible  in  fewer  manpower  demands,  that  is,  a 
highly  reliable  system  requires  fewer  maintenance 
man-hours.  Reliability  improvements  increase  the 
idle  times  for  the  particular  AFSC  technician 
responsible  for  repairs.  Evaluation  of  both  the  R&M 
characteristics  of  the  system,  together  superimposed 
on  a  graph  with  common  ordinates  (i.e.,  total  flying 
hours),  can  help  assess  and  present  an  overview 
picture  of  the  M&P  demands  imposed  on  the  aircraft 
system.  The  M&P  element  can  be  directly  related  to 
reliability  by  applying  the  following  formula: 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  in  section  3.4.1 

LTM  21  -  Maintainability 

Direct  Maintenance  Man-hoiirs  per  Flight  Hour. 
DMMH/FH  is  described  in  the  section  3.4.1. 

LTM  22  Average  Crew  Size 

Average  Crew  Size.  Crew  size  refers  to  the  number 
of  maintenance  personnel  used  to  perform  a  specific 
maintenance  task.  The  average  crew  size  represents 
the  average  number  of  maintenance  personnel 
utilized  for  accomplishing  a  specific  aircraft 
maintenance  task(s).  Maintenance  man-hours 
(MMHs),  together  with  elapsed  repair  times  (ET)  in 
hours  required  to  complete  each  maintenance  task, 
can  be  used  to  calculate  average  crew  size  numbers. 

Maintenance  Man-Hours  (MMHs).  Is  a  measure  of 
repair  times  which  considers  the  number  MMHs 
expended  in  the  repair  of  the  aircraft  system. 
Maintenance  man-hours  can  be  determined  by 
evaluating  the  total  MMHs  utilized  for  a  specific 
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Type  1  (inherent),  Type  2  (induced),  or  Type  6  (NO 
DEFECT,  CND,  or  FOM)  maintenance  action.  The 
following  formula  applies: 

Formula  6: 


MMHt  =  ^  MMH  (MAN-HOURS) 


where: 

MMHt  is  the  sum  total  of  all  maintenance  man-hours 
consumed  for  a  specific  Type  1, 
Type  2,  or  Type  6  maintenance  action/task 
accomplished. 

Elapsed  Times  (ET).  Is  a  measure  of  FT  which 
considers  the  actual  repair  times  in  hours  expended 
while  performing  a  specific  aircraft  maintenance 
task.  Elapsed  times  can  be  determined  by  summing 
the  total  repair  hours  for  each  Type  1,  Type  2,  or 
Type  6  maintenance  task.  The  following  formula 
applies: 

Formula  7: 


ETt  =  ^ETi  (HOURS) 


where: 

ETt  is  the  sum  total  of  all  maintenance  hours 
expended  for  a  specific  Type  1,  Type  2,  or  Type  6 
maintenance  task  accomplished  on  the  system  during 
test. 

Knowing  MMHt  and  ETt,  an  average  crew  size  for  a 
specific  task  can  be  determined  by  using  the 
following  formula: 

Formula  8: 


MMH  {rmn-hourf^ specific  task) 

AVER4CE  CREWSZE= - - 

ETt  {hours/ specific  task) 


where: 

average  crew  size  represents  the  average  number  of 
maintenance  personnel  utilized  for  accomplishing  a 
specific  aircraft  maintenance  task(s). 

LTM  23  -  Air  Force  Specialty  Code  (AFSC) 

LTM  23  identifies  the  percent  of  specific  Air  Force 
maintenance  specialties  and  skill  levels  used  to 
accomplish  a  specific  task(s).  Air  Force  Specialty 
Code  data  can  be  determined  directly  from  SEDS. 
Skill  levels  can  also  be  determined  from  the 
maintenance  event  listings  by  extracting  the  fourth 
digit  of  the  AFSC  code. 


Air  Force  Specialty  Code  (AFSC).  Is  defined  as  a 
grouping  of  five  digit  numbers  identifying  special 
positions  within  the  Air  Force  (AFP  36-2241, 
Volume  1,  Reference  4).  The  objective  of  this 
measure  is  to  identify  what  percentage  of  a  specific 
task(s)  is  being  accomplished  by  a  specific  Air  Force 
specialty.  For  example,  70  percent  of  the  towing 
task  is  being  accomplished  by  crew  chiefs  (2A5X1). 
The  following  formula  can  be  used  to  derive  the 
AFSCs: 


Formula  9: 


where: 


AFSC  identifies  the  percentage  of  a  specific  task(s). 
Type  1,  Type  2  or  Type  6,  accomplished  by  a 
specific  Air  Force  specialty. 

AFSCi  can  be  determined  by  summing  the  number 
of  personnel  within  a  given  AF  specialty  (i.e., 
2A6X5)  used  to  perform  a  specific  maintenance 
action/task  (Type  1,  Type  2  or  Type  6).  AFSCt  is  the 
summation  of  all  Air  Force  specialties  used  to 
accomplish  a  specific  maintenance  task  (Type  1, 
Type  2  or  Type  6). 

4.4.2  Measurement  Criteria 

LTM  20  -  Reliability 

All  maintenance  events  will  be  evaluated 

individually  and  as  groups  (i.e.,  LCNs)  to  determine 
the  M&P  requirements  based  upon  their  reliability. 
Those  maintenance  events  or  groups  of  maintenance 
events  which  do  not  meet  their  allocated 
maintainability  will  receive  top  consideration  in  this 
evaluation. 

LTM  21  -  Maintainability 

All  maintenance  events  will  be  evaluated 

individually  and  as  groups  (i.e.,  LCNs)  to  determine 
the  M&P  requirements  based  upon  their 

maintainability.  Those  maintenance  events  or  groups 
of  maintenance  events  which  do  not  meet  their 
allocated  maintainability  will  receive  top 

consideration  in  this  evaluation. 

LTM  22  -  Average  Crew  Size 

The  crew  size  requirements  contained  within  the 
aircraft  LSAR  database  will  be  used  for  evaluation 
purposes.  Specifically,  the  data  contained  in  the 
LSA-075  Manprint,  LSA-001  Annual  Maintenance 
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Man-Hours  (by  skill  specialty  code  and  level  of 
maintenance),  and  the  LSA-002  Personnel  and  Skill 
Summary  Records,  will  be  used  for  the  evaluation. 
The  LSA  records  contain  detailed  M&P  information 
for  each  maintenance  action  to  be  performed  in  the 
aircraft.  Actual  crew  size  calculations  can  be 
determined  directly  from  the  LTM  formulas 
described  in  section  4.4.1  and  the  SEDS  data. 

LTM  23  -  Air  Force  Specialty  Code 

The  AFSC  requirements  contained  within  the  aircraft 
LSAR  database  will  be  used  for  evaluation  purposes. 
Specifically,  the  data  contained  in  the  LSA-075 
Manprint,  LSA-001  Annual  Maintenance  Man-Hours 
(by  skill  specialty  code  and  level  of  maintenance), 
and  the  LSA-002  Personnel  and  Skill  Summary 
Records,  will  be  used  for  the  evaluation.  The  LSA 
records  contain  detailed  M&P  information  for  each 
maintenance  action  to  be  performed  in  the  aircraft. 
Actual  AFSC  calculations  can  be  determined  directly 
from  the  LTM  formulas  described  in  section  4.4.1 
and  the  SEDS  data. 

4.4.3  Analysis  Techniques  and  Tools 

An  evaluation  of  the  results  derived  from  the  above 
referenced  LTMs  will  be  conducted  to  provide 
projections  and  detailed  characteristics  of  the  M&P 
element.  For  example,  the  following  evaluations  will 
be  applied. 

•  Project  the  number  of  AFSCs  needed  at  the  first 
operational  unit.  The  projections  will  be  based 
on  the  accumulated  data  for  each  AFSC  that  was 
used  in  the  performance  of  maintenance  actions, 
for  a  period  of  time,  at  the  CTF.  For  example, 
the  number  of  man-hours,  and  AFSCs  expended 
for  accomplishing  preventive  and  corrective 
maintenance  actions  for  a  specified  period  of 
time  at  the  CTF  (i.e.,  yearly  rates),  will  be  used 
for  projecting  the  M&P  requirements  for  the 
first  unit.  The  information  for  making  the 
estimates  will  be  based  on  data  collected 
through  SEDS. 

4.5  Human  Factors 

The  following  LTM  influences  the  M&P  element 
and  is  addressed  in  section  4.5.1 : 

LTM  24  -  Human  Performance 

4.5.1  Evaluation  Measures 

Logistics  test  measure  24  will  focus  on  the  ability  of 
maintenance  personnel  to  perform  tasks  on  the 


aircraft  under  all  conditions  using  appropriate 
regulations  and  T.O.s. 

LTM  24  -  Human  Performance 

Human  Performance  refers  to  evaluation  of  the 
human/machine  interface  in  the  environment  in 
which  the  system  will  be  operated  to  include 
temperature  extremes,  high  humidity,  and 
precipitation.  Rate  the  ability  to  perform  the  task  in 
all  environments  while  wearing  personnel  protective 
equipment. 

4.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

4.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

4.6  Logistics 

The  following  LTMs  influence  the  M&P  element 
and  are  addressed  in  section  4.6.1 : 

LTM  25  -  Crew  Size 

LTM  26  -  Air  Force  Specialty  Code  (AFSC)  and 
Skill  Level 

4.6.1  Evaluation  Measures 

Logistics  test  measures  25  and  26  will  be  evaluated 
to  determine  the  optimum  quantity  of  personnel  and 
AFSCs  required  to  accomplish  maintenance/ support 
tasks  for  the  aircraft  and  related  systems. 

LTM  25  -  Crew  Size  Adequacy 

Crew  Size  Adequacy  refers  to  the  technical  order 
data  (TOD)-specified  number  of  personnel  required 
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to  facilitate  task  completion.  This  is  designed  to 
evaluate  whether  all  manpower  was  effectively 
utilized;  whether  there  were  enough  people  assigned 
to  ensure  safe  performance  of  the  task;  and  if 
additional  manpower  is  required.  Rate  the  adequacy 
of  the  TOD-specified  number  of  personnel  required 
to  perform  the  task. 

LTM  26  -  Air  Force  Specialty  Code  (AFSC)  and 
Skill  Level 

Air  Force  Specialty  Code  and  skill  level  is  a  5-digit 
number  identifying  specialty  positions  within  the  Air 
Force.  This  is  designed  to  indicate  whether  or  not 
the  appropriate  AFSC(s)  and  skill  level(s)  have  been 
assigned  to  the  task.  Rate  the  adequacy  of  the  TOD- 
identified  AFSC(s)  and  skill  level(s)  required  to 
accomplish  the  maintenance  task. 

4.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 


4.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

4.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single-rating  conclusion  for  the 
ILS  element  based  on  mission  impact.  The  M&P 
element  directly  impacts  Maintenance  Planning  and 
will  be  taken  into  consideration  when  making  an 
overall  assessment  of  these  elements. 
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5.0  SUPPORT  EQUIPMENT  (SE) 

5.1  Description 

Support  equipment  (SE)  includes  all  equipment 
required  to  perform  the  support  functions  of  the 
aircraft  system,  except  those  which  are  an  integral 
part  of  the  mission  equipment.  Support  equipment 
includes  all  tools,  special  monitoring  equipment, 
diagnostic  and  checkout  equipment,  associated 
multi-use  end  items,  ground  handling  and 
maintenance  equipment,  repair  kits,  calibration 
equipment,  manual  and  automated  test  equipment 
(ATE),  support  equipment  for  on-  and  off-equipment 
maintenance,  and  any  related  computer  equipment 
and  software  programs  required  to  support  scheduled 
and  unscheduled  maintenance  actions.  This  element 
also  includes  SE  used  during  depot  testing  and 
manufacturing  that  supports  the  production  and/or 
depot  repair  of  the  end  items  or  components. 

5.2  Objective 

The  purpose  of  this  evaluation  is  to  assure  that  the 
planned  SE,  both  contractor-  and 
government-furnished  (CFE  and  GFE),  effectively 
support  the  aircraft  and  associated  systems.  The 
evaluation  will  also  determine  if  the  SE  is  capable  of 
supporting  maintenance  through  functionality, 
compatibility,  ease  of  use  and  safety;  identify  SE 
supportability  problem  areas  and  suggest  possible 
corrective  actions;  and  identify  additional  or 
alternative  SE  requirements.  Any  additional  SE  that 
becomes  available  during  flight  test  or  that  was  not 
included  in  the  initial  planning  will  also  be 
evaluated. 

5.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  SE 
element: 

Reliability  &  Maintainability 

LTM  30  -  Reliability 
LTM  3 1  -  Maintainability 

Human  Factors 

LTM  32  -  Ease  of  Use 
LTM  33  -  Handling 
LTM  34  -  Safety 

Logistics 

LTM  35  -  Functionality 
LTM  36  -  Setup/Disassembly 


5.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  SE  element  and  are  addressed  in  detail 
in  section  5.4.1. 

LTM  30  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

Mean  Time  Between  Removal  —  MTBR 

LTM  31  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour  — 
DMMH/FH 

Mean  Man-Hours  to  Repair  -  MTTR 

5.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  the  LTMs  cited  in  section  5.3  to  SE. 

LTM  30  -  Reliability 

The  aircraft  system  reliability  will  be  evaluated 
under  this  LTM.  This  LTM  evaluates  the  optimal 
reliability  performance  characteristics  of  the  aircraft 
system.  The  objective  is  to  determine  if  the  planned 
reliability  and  ILS  criteria  have  been  met. 

Mean  Time  Between  Maintenance  (Inherent). 
MTBM(I)  is  described  in  section  3.4.1. 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  in  section  3.4.1. 

Mean  Time  Between  Removal.  MTBR  is  described 
in  section  3.4.1. 

LTM  31  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour. 
DMMH/FH  is  described  in  section  3.4.1. 

Mean  Time  to  Repair.  MTTR  is  described  in 
section  3.0. 

5.4.2  Measurement  Criteria 

Maintenance  tasks  which  require  SE  will  be 
evaluated  to  determine  what  amount  of  SE  will  be 
needed  to  support  the  number  of  maintenance  task. 
Those  maintenance  tasks  whose  frequency  is  more 
than  expected  will  be  the  first  candidates  for  such  an 
analysis. 
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5.4.3  Analysis  Techniques  and  Tools 

Actual  values  for  the  R&M  metrics  will  be  obtained 
from  SEDS. 

5.5  Human  Factors 

The  following  LTMs  will  be  evaluated  by  Human 
Factors  for  the  SE  element. 

LTM  32  -  Ease  of  Use 
LTM  33  -  Handling 
LTM  34  -  Safety 

5.5.1  Evaluation  Measures 

Logistics  test  measures  32,  33,  and  34  will  be 
evaluated  to  determine  the  ability  of  the  SE  to  be 
utilized  effectively  without  risking  personnel  injury, 
equipment  damage,  or  excessive  fatigue. 

LTM  32 -Ease  of  Use 

Ease  of  use  refers  to  the  ability  to  utilize  SE  with 
minimal  physical  and  mental  workload.  This  is 
designed  to  evaluate  the  SE  from  the  aspect  of 
adequate  access  space  for  interfacing  SE  with  the 
aircraft,  SE  visual  and  other  communication  links 
with  personnel,  efficient  arrangement  of  controls  and 
displays,  and  compatibility  with  personnel  protective 
equipment  requirements  under  various 
environmental  conditions.  Rate  the  adequacy  of  the 
SE  in  relation  to  adequate  access  space  for 
interfacing  SE  with  the  aircraft,  SE  visual  and  other 
communication  links  with  personnel,  efficient 
arrangement  of  controls  and  displays,  and 
compatibility  with  personnel  protective  equipment 
under  various  environmental  conditions. 

LTM  33  -  Handling 

Handling  refers  to  the  ability  to  move  or  transport  the 
SE.  This  is  designed  to  evaluate  the  adequacy  of 
provisions  for  handles  or  other  suitable  means  for 
grasping,  handling,  and  carrying  support  equipment; 
considering  biodynamics  and  weight  limits  for 
moving  of  equipment.  Rate  the  adequacy  of  the  SE 
in  relation  to  handling/grasping  surfaces  provided 
and  ease  of  transporting  (either  by  hand-carry  or  by 
towing). 

LTM  34  -  Safety 

Safety  refers  to  SE  design  features  and  instructions 
that  minimize  the  risk  of  personnel  injury  or 
equipment  damage  in  any  maintenance  environment. 
This  is  designed  to  evaluate  the  adequacy  of  existing 
safety  features.  Identify  potentially  hazardous 
conditions  requiring  safety  features  and  ensure  the 
proper  placement  of  warnings,  cautions,  and  notes. 


Rate  the  adequacy  of  the  SE  design  features  and 
instructions  for  minimizing  the  risk  of  personnel 
injury  or  equipment  damage  in  any  maintenance 
environment. 

5.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

5.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

5.6  Logistics 

The  following  factors  influence  the  SE  element  and 
are  addressed  in  detail  in  section  5.6.1; 

LTM  35  -  Functionality 
LTM  36  -  Durability 
LTM  37  -  Setup/Disassembly 

5.6.1  Evaluation  Measures 

Logistics  test  measures  35,  36,  and  37  will  be 
evaluated  to  determine  the  adequacy  of  the  planned 
SE,  both  contractor-  and  government-furnished  (CFE 
and  GFE),  in  relation  to  effectively  support  the 
aircraft  and  associated  systems. 

LTM  35  -  Functionality 

Functionality  refers  to  the  form,  fit,  and  function  of 
the  SE  and  its  ability  to  operate  as  per  design.  This 
is  designed  to  evaluate  the  adequacy  of  the  SE  in 
relation  to  the  ability  to  position  and  adjust  it  at  a 
safe,  comfortable,  and  useable  distance  from  the 
weapon  system  as  well  as  perform  its  required 
function  as  it  was  designed  to.  Rate  the  adequacy  of 
the  SE  in  relation  to  positioning,  adjustment,  and 
performing  its  designated  function. 
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LTM  36  -  Durability 

Durability  refers  to  the  resistance  to  wear  of  the 
specific  piece  of  support  equipment.  This  is 
designed  to  evaluate  the  durability  of  the  SE,  which 
impacts  the  SE  availability.  Rate  the  adequacy  of  the 
SE  in  relation  to  its  durability. 

LTM  37  -  Setup/Disassembly 

Setup/Disassembly  refers  to  SE  design  features  that 
promote  ease  and  economy  of  maintenance  in 
normal,  adverse,  and  emergency  maintenance 
environments.  This  is  designed  to  evaluate  the 
adequacy  of  the  SE  in  relation  to  setup  and 
disassembly  times.  Support  equipment  that  requires 
an  unreasonable  amount  of  time  to  setup  or 
disassemble  will  have  a  negative  impact  on  the 
operational  availability  of  the  aircraft.  Rate  the 
adequacy  of  the  SE  in  relation  to  ease  and  economy 
of  maintenance  considering  setup  and  disassembly 
time. 

5.6.2  Evaluation  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logisties  test 
measures  rated  4  or  below,  or  having  signifieant 
negative  comments,  will  be  analyzed  in  aceordanee 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 


5.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

5.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single  rating  conclusion  for  the 
ILS  element  based  on  mission  impact. 

The  SE  element  directly  impacts  MP  and  DI,  and 
will  be  taken  into  consideration  when  making  the 
overall  assessments  of  these  elements. 
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6.1  Description 

Supply  support  (SS)  is  that  provisioning  process  and 
effort  which  provides  timely  and  adequate  spare 
parts,  components,  items  and  equipment,  and  specific 
supplies  to  satisfy  operation  and  maintenance 
functions  during  the  life  cycle  of  the  weapon  system. 
Supply  support  encompasses  all  management 
actions,  processes,  procedures,  and  techniques 
necessary  to  acquire,  catalog,  receive,  store,  transfer, 
issue,  and  dispose  of  aircraft-related  parts, 
components,  items  and  equipment.  In  simple  terms, 
supply  support  is  that  provisioning  process  and  effort 
that  ensures  that  maintenance  personnel  has  the  right 
part,  component,  items  and  equipment  at  the  right 
time,  in  the  right  place,  in  the  right  quantity,  at  the 
most  economical  cost. 

Provisioning  is  a  process  or  series  of  supply  support 
actions  that  extend  over  a  wide  range  of  functions, 
including  design,  MP,  supply,  requirements 
determination,  item  entry  control,  procurement, 
cataloging,  and  control  administration. 

6.2  Objective 

To  evaluate  and  ensure  that  the  planned  supply 
provisioning  process  and  efforts  effectively  support 
the  aircraft  system.  The  goal  is  to  quantitatively  and 
qualitatively  evaluate  the  planned  supply 
provisioning  process  and  support,  and  report  the 
results.  This  will  include  identifying  the  supply 
support  deficiencies  and  possible  corrective  actions, 
and  identifying  additional  and/or  alternative  supply 
support  requirements. 

6.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  SS 
element: 

Reliability  and  Maintainability 

LTM  40  -  Reliability 

Hnman  Factors  —  None 
Logistics 

LTM  41  -  Source,  Maintenance,  and  Recoverability 

(SMR)  Codes 

LTM  42  -  Availability 

6.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  SS  element  and  are  addressed  in  detail 
in  section  6.4.1. 


LTM  40  -  Reliability 

Mean  Time  Between  Removal  —  MTBR 

6.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  LTMs  cited  in  section  6.3  to  SS. 

LTM  40  -  Reliability 

The  aircraft  system  reliability  will  be  evaluated 
under  this  LTM.  This  LTM  evaluates  the  optimal 
reliability  performance  characteristics  of  the  aircraft 
system.  The  objective  is  to  determine  if  the  planned 
reliability  and  ILS  criteria  have  been  met. 

Mean  Time  Between  Removal.  MTBR  is  addressed 
in  section  3.4.1. 

6.4.2  Measurement  Criteria 

Data  collected  will  be  analyzed  and  comparisons 
made  of  the  relative  demands  made  on  the  supply 
system  for  each  piece  of  equipment  removed  or 
replaced  within  a  specified  maintenance  action. 
Collected  MTBR  data  will  help  to  correlate  and  draw 
conclusions  about  the  demands  made  on  the  aircraft 
supply  system. 

6.4.3  Analysis  Techniques  and  Tools 

Actual  values  for  the  R&M  metrics  will  be  obtained 
from  SEDS.  These  values  will  be  compared  to  the 
corresponding  values  contained  within  the  LSAR 
database.  All  data  required  to  compute  these  metrics 
will  be  stored  within  SEDS.  In  addition,  the 
corresponding  values  from  the  LSAR  database  will 
be  stored  in  SEDS,  which  will  aid  in  the  comparison 
process. 

Maintenance  event  data  for  LTM  40  can  also  be 
broken  down  to  the  system  and  subsystem  level  and 
specified  in  SEDS  listings.  Mean  time  between 
removal  can  also  be  specified,  if  designed,  in  terms 
of  specific  HMC  data  such  as  070  -  broken;  105  - 
loose,  damaged,  or  missing  hardware;  254  -  no 
output;  949  -  computer  memory  error,  etc.,  which  are 
specified  in  the  SEDS  maintenance  event  listings. 
Other  useful  information  which  is  also  extracted 
from  the  SEDS  maintenance  listings  include  (a) 
information  about  failed  equipment  such  as 
manufacturer's  name  (MEG),  noun,  serial  number, 
and  part  number.;  (b)  information  about  the  installed 
equipment  such  as  manufacturer's  name  (MEG), 
noun,  serial  number,  and  part  number.;  and  (c) 
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discrepancy  information  about  the  equipment 
repaired  and  replaced  such  as  piece  parts  data. 

6.5  Logistics 

The  following  factors  influence  the  SS  element  and 
are  addressed  in  detail  in  section  6.5.1: 

LTM  41  -  Source,  Maintenance,  and  Recoverability 

(SMR)  Codes 

LTM  42  -  Availability 

6.5.1  Evaluation  Measures 

Logistics  test  measures  41  and  42  will  be  evaluated 
to  assure  that  the  planned  supply  provisioning 
process  and  efforts  effectively  support  the  aircraft 
system. 

LTM  41  -  Source,  Maintenance,  and  Recoverability 
(SMR)  Codes 

The  source  code  identifies  where  the  item  is 
procured  from  (i.e.,  depot,  local  purchase,  local 
manufacture).  The  maintenance  code  provides  the 
repair  level  (i.e.,  organization  or  depot)  for  each 
aircraft  part.  The  recoverability  code  provides 
information  of  how  a  part/component  is  to  be 
disposed.  Maintenance  and  recoverability  are  the 
primary  codes  to  be  evaluated  under  this  LTM.  This 
is  designed  to  assess  the  appropriateness  of  the  SMR 
code  identified  in  the  illustrated  parts  breakdown 
(IPB).  Rate  the  appropriateness  of  the  SMR  code 
identified  in  the  IPB  (when  performing  removal  tasks 
on  equipment  and  repair  tasks  off  equipment). 

LTM  42  -  Availability 

Availability  refers  to  having  the  required  aircraft 
parts  and  consumables  on  hand  to  minimize  aircraft 
downtime  or  degrade  mission  effectiveness.  This  is 
designed  to  identify  deficiencies  when  attempting  to 
acquire  spares,  consumable  bench  stock  items,  or 
suitable  subs  through  the  supply  system.  Rate  the 
availability  of  spares,  consumable  bench  stock  items, 
and  suitable  subs  through  the  supply  system. 


6.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

6.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

6.6  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single  rating  conclusion  for  the 
ILS  element  based  on  mission  impact. 

The  SS  element  directly  impacts  MP  and  DI  and  will 
be  taken  into  consideration  when  making  the  overall 
assessment  of  these  element. 
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7.0  TECHNICAL  DATA  (TD) 

7.1  Description 

Technical  data  (TD)  refers  to  recorded  information 
used  to  translate  system  requirements  into 
engineering  and  logistic  considerations.  Technical 
data  is  needed  to  develop,  support,  operate  and 
maintain  the  weapon  system.  Technical  data 
includes  T.O.s,  technical  manuals  (TMs)  and 
engineering  data  (ED).  The  TD  logistic  element 
concerns  itself  with  the  evaluation  of  validated  and 
verified  T.O.s,  TMs,  and  ED  used  in  the 
maintenance,  operation  and  support  of  the  aircraft 
system.  Technical  orders  and  TMs  include  flight 
manuals,  operator  manuals,  maintenance 
instructions,  illustrated  parts  breakdown,  and  time 
compliance  manuals  required  to  operate,  maintain, 
service,  repair,  overhaul  and  modify  the  system, 
subsystems,  or  equipment  acquired  to  support  the 
aircraft.  Engineering  data  is  recorded  information 
regardless  of  form  or  character  of  engineering  or 
technical  nature.  Engineering  data  includes 
specifications,  drawings,  test  procedures,  and 
computer  programs  which  guide  engineering 
personnel  in  performing  operations  and  supporting 
aircraft  tasks. 

Validation  is  the  process  by  which  contractor 
personnel  test  TD  for  technical  accuracy. 
Verification  is  the  process  in  which  TD  is  tested  and 
approved  by  Government  personnel  for  ease  of  use, 
understanding,  adequacy  of  operation,  maintenanee, 
and  support  of  systems,  subsystems  or  equipment 
acquired  for  support  of  the  test  aircraft. 

While  the  Human  Factors  evaluation  of  technical 
data  development  is  important  to  logistics  test,  it  is 
understood  that  technical  documentation  is  in  an 
early  stage  and  will  be  improving  continuously 
through  the  test  program  and,  in  fact,  throughout  the 
life  of  the  aircraft.  Conclusions  arrived  at  during 
logistics  test  will  represent  a  ‘snap  shot’  or  end  of 
initial  development  of  TD  as  opposed  to  fixed 
conclusions  as  found  in  other  areas;  for  example,  the 
SE  element.  The  very  fact  that  TD  is  constantly 
being  revised  highlights  one  of  the  features  of  a 
performance  aid.  Performance  aids  are  easier  to 
revise  when  procedures  or  equipment  change  than 
the  maintenance  person's  learning  already 
established  by  training. 

7.2  Objective 

To  evaluate  the  adequacy  and  ability  of  TD  provided 
by  the  contractor  for  performing  maintenance  tasks 
on  the  aircraft.  The  evaluation  will  focus  on  an 
overall  assessment  of  the  adequacy  and  ability  of  the 
TD  to  serve  as  a  performance  aid  to  the  technician 


during  maintenance  and  support  of  the  aircraft 
system. 

7.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  TD 
element. 

Reliability  and  Maintainability 

LTM  50  -  Reliability 
LTM  5 1  -  Maintainability 

Hnman  Factors 

LTM  52  -  Safety 

LTM  53  -  Effectiveness  of  Instruction 

Logistics  Test 

LTM  54  -  Task  Requirements 
LTM  55  -  Reference  Material 

7.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  TD  element  and  are  addressed  in  detail 
in  section  7.4.1. 

LTM  50  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

Mean  Time  Between  Removal  -  MTBR 

LTM  51  -  Maintainability 

Maintenance  Man-Hours  per  Flight  Hour  — 

MMH/FH 

Mean  Man-Hours  to  Repair  —  MMTR 

7.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  the  LTM  cited  in  section  7.3  to  TD. 

LTM  50  -  Reliability 

The  aircraft  system  reliability  will  be  evaluated 
under  this  LTM.  This  LTM  evaluates  the  optimal 
reliability  performance  characteristics  of  the  aircraft 
system.  The  objective  is  to  determine  if  the  planned 
reliability  and  ILS  criteria  have  been  met. 

Mean  Time  Between  Maintenance  (Inherent). 
MTBM(I)  is  described  in  section  3.4.1. 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  in  section  3.4.1. 

Mean  Time  Between  Removal.  MTBR  is  described 
in  section  3.4.1. 
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LTM  51  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour. 
DMMH/FH  is  described  in  section  3.4.1. 

Mean  Time  to  Repair.  MTTR  is  described  in  section 
3.4.1. 

7.4.2  Measurement  Criteria 

Maintenance  tasks  which  require  TD  to  properly 
perform  the  maintenance  will  be  included  in  this 
evaluation.  Reliability  and  maintainability  metrics 
will  aid  in  determining  the  actual  frequencies  in 
comparison  to  predicted  frequencies  contained  in  the 
LSAR,  and  actual  task  time  and  crew  size  required  to 
perform  the  maintenance  task  in  comparison  to 
predicted  values  contained  in  the  TD. 

7.4.3  Analysis  Techniques  and  Tools 

Actual  values  for  the  R&M  metrics  will  be  obtained 
from  SEDS. 

7.5  Human  Factors 

The  following  LTMs  influence  the  TD  element  and 
are  addressed  in  section  7.5.1. 

LTM  52  -  Safety 

LTM  53  -  Effectiveness  of  Instruction 

7.5.1  Evaluation  Measures 

Logistics  test  measures  52  and  53  will  be  evaluated 
to  determine  the  adequacy  and  ability  of  TOD  to 
provide  instructions  that  are  clear,  concise,  and  in  a 
logical  sequence  while  minimizing  risk  to  personnel 
and  equipment. 

LTM  52  -  Safety 

Safety  refers  to  instructions  that  minimize  the  risk  of 
personnel  injury  or  equipment  damage  in  any 
maintenance  environment.  This  is  designed  to 
evaluate  the  adequacy  of  TD  procedures  for  proper 
placement  of  warnings,  cautions,  and  notes. 

LTM  53  -  Effectiveness  of  Instructions 

Effectiveness  of  instructions  refers  to  the  capability 
of  TD  instructions  to  aid  in  performing  a  task.  This 
is  designed  to  evaluate  the  adequacy  of  TD 
instructions  considering  the  simplicity,  accuracy,  and 
relative  ease  of  understanding  in  the  performance  of 
the  task. 

7.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 


significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

7.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  valuations. 

7.6  Logistics 

The  following  LTMs  influence  the  TD  element  and 
are  addressed  in  section  7.6.1. 

LTM  54  -  Task  Requirements 
LTM  55  -  Reference  Material 

7.6.1  Evaluation  Measures 

Logistics  test  measures  54  and  55  will  be  evaluated 
to  determine  the  adequacy  and  ability  of  TOD  in 
performing  maintenance  tasks  on  the  aircraft. 

LTM  54  -  Task  Requirements 

Task  requirements  refers  to  the  personnel, 
provisions,  and  preparatory  tasks  necessary  to 
facilitate  task  completion.  This  is  designed  to 
evaluate  the  adequacy  of  TOD  identification  of  (a) 
the  existence  of  required  procedures  and  (b)  the 
adequacy  of  the  identification  of  task-related 
requirements  (i.e.,  pre-existing  conditions,  proper 
tools,  SE,  consumables,  AFSC(s),  skill  level(s),  and 
other  T.O.s  for  use  during  the  same  task).  Rate  the 
existence  of  required  procedures  and  the  adequacy  of 
the  identification  of  task-related  requirements  (i.e., 
pre-existing  conditions,  proper  tools,  SE, 
consumables,  AFSC(s),  skill  level(s),  and  other  T.O.s 
for  use  during  the  same  tasks). 

LTM  55  -  Reference  Material 

Reference  material  refers  to  the  supplemental  data 
used  in  conjunction  with  technical  procedures  to 
facilitate  task  completion.  This  is  designed  to 
evaluate  the  existence,  adequacy  and  accuracy  of 
drawings,  figures,  and  schematic/wiring  diagrams 
required  to  facilitate  task  completion.  Rate  the 
adequacy  of  the  drawings,  figures,  and 
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schematic/wiring  diagrams  for  existence,  accuracy, 
and  value  as  a  fault  isolation  aid. 

7.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

7.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 


excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airirame,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

7.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single  rating  conclusion  for  the 
ILS  element  based  on  mission  impact.  In  order  to 
make  an  accurate  assessment  of  the  TD  element,  the 
impacts  on  M&P,  SE,  and  SS  elements  have  to  be 
taken  into  consideration. 
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8.0  TRAINING  AND  TRAINING  SUPPORT  (TTS) 


8.1  Description 

Training  and  training  support  (TTS)  is  the  process, 
procedures,  techniques,  and  equipment  used  to 
instruct  personnel  to  operate  and  maintain  the 
weapon  system.  The  training  program  is  a  structured 
effort  outlined  in  accordance  with  the  guidance  of 
Air  Force  Instruction  (AFl)  36-2201  (Reference  5) 
which  provides  the  guidelines  to  train  both  skilled 
and  unskilled  technicians  to  successfully  perform 
aircraft  maintenance  tasks.  This  training  program  is 
composed  of  both  formal  and  informal  training. 
Formal  training  consists  of  classroom,  observation, 
and  hands-on  instruction.  Hands-on  instruction 
makes  use  of  maintenance  training  devices  (i.e., 
engineering  devices/simulators/mockups)  and  the 
actual  air  vehicle.  Infomial  training  consists  of 
on-the-job  training  (OJT)  designed  to  qualify 
maintenance  technicians  to  perform  maintenance 
tasks.  Training  will  be  a  continuous  process 
throughout  the  test  program  to  include  individual 
training,  new  equipment  training,  formal,  and  OJT. 

8.2  Objective 

The  purpose  of  the  training  evaluation  is  to  evaluate 
the  effectiveness  of  the  training  program  established 
to  support  the  aircraft.  Also,  to  determine  if  all 
training  requirements  have  been  adequately 
identified  and  the  training  program  is  of  sufficient 
scope  to  meet  day-to-day  aircraft  generation  and 
contingency  requirements. 

The  evaluation  will  assess  the  adequacy  of  Type  1 
(contract/factory)  and  Type  4  (onsite/mobile) 
training;  identify  training  deficiencies  and  possible 
corrective  actions;  and  identify  additional  or 
alternative  training  requirements.  Additionally,  the 
evaluation  is  designed  to  determine  if  maintenance 
personnel  are  being  provided  the  adequate 
knowledge  and  proficiency  training  to  safely 
accomplish  maintenance  tasks.  Through  these 
evaluations,  training  shortfalls  and  corrective  actions 
will  be  identified. 

8.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the 
TTSelement: 

Reliability  and  Maintainability 

LTM  60  -  Reliability 
LTM  61  -  Maintainability 


Human  Factors 

LTM  62  -  Safety 

Logistics 

LTM  63  -  Knowledge  Training  -  Type  1 
LTM  64  -  Knowledge  Training  -  Type  4 
LTM  65  -  Proficiency  Training 

8.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  TTS  element  and  are  addressed  in  section 
8.4.1; 

LTM  60  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

Mean  Time  Between  Removal  —  MTBR 

LTM  61  -  Maintainability 

Task  Times 
Crew  Size 

8.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  LTMs  cited  in  section  8.3  to  TTS. 

LTM  60  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent). 
MTBM(I)  is  described  in  section  3.4.1. 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  in  the  section 
3.4.1. 

Mean  Time  Between  Removal.  MTBR  is  described 
in  the  section  3.4.1. 

LTM  61  -  Maintainability 

Task  Completion  Times.  Task  Completion  Times  is 
described  in  the  section  3.6.1. 

Average  Crew  Size.  Average  Crew  Size  is  described 
in  the  section  4.4.1. 

8.4.2  Measurement  Criteria 

Maintenance  task  which  do  not  match  the  training 
received  by  the  maintainer  will  be  evaluated  under 
this  element.  The  impact  and  scope  of  this 
evaluation  has  to  be  determined. 
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8.4.3  Analysis  Techniques  and  Tools 

The  SEDS  will  provide  actual  maintenance 
man-hours,  repair  times,  task  frequencies,  and  flight 
hours  to  compute  the  R&M  metrics.  When  a 
deficient  training  procedure  is  identified,  R&M 
metrics  will  be  used  to  provide  quantitative  data  as  to 
the  impact  of  the  deficient  training  procedure. 

8.5  Human  Factors 

The  following  LTM  influence  the  TTS  element  and 
is  addressed  in  section  8.5.1: 

LTM  62  -  Safety 

8.5.1  Evaluation  Measures 

Logistics  test  measure  62  will  be  evaluated  to 
determine  the  effectiveness  of  the  training  program 
and  to  ensure  all  safety-related  training  requirements 
have  been  adequately  identified. 

LTM  62  -  Safety 

Safety  refers  to  the  ability  of  maintenance  personnel 
to  minimize  task  of  personal  injury  and/or  damage  to 
equipment  while  accomplishing  various  tasks  on  or 
around  the  aircraft.  In  order  for  training  to  be 
satisfactory,  it  must  address  safety  issues  pertinent  to 
the  task.  This  is  designed  to  evaluate  the  existence 
and  adequacy  of  task  pertinent  safety  training.  Rate 
the  adequacy  of  task  pertinent  safety  training. 

8.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

8.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 


8.6  Logistics 

The  following  LTMs  influence  TTS  element  and  are 
addressed  in  section  8.6.1: 

LTM  63  -  Knowledge  Training  (Type  1) 

LTM  64  -  Knowledge  Training  (Type  4) 

LTM  65  -  Proficiency  Training 

8.6.1  Evaluation  Measures 

Logistics  test  measures  63,  64,  and  65  will  be 
evaluated  to  determine  the  effectiveness  of  the 
training  program  established,  to  ensure  all  training 
requirements  have  been  adequately  identified,  and  to 
determine  if  the  training  program  is  of  sufficient 
scope  to  meet  day-to-day  aircraft  generation  and 
contingency  requirements. 

LTM  63  -  Knowledge  Training  (Type  1) 

Knowledge  Training  (Type  1)  refers  to 
contractor-furnished  training  provided  to  give 
maintenance  personnel  a  basic  understanding  of  the 
theory  of  operation  of  their  particular  systems. 
Knowledge  training  also  involves  developing  sound 
troubleshooting  skills  and  preparing  maintenance 
personnel  for  OJT.  This  is  designed  to  evaluate  the 
adequacy  of  the  theory  of  operation  training  to 
prepare  maintenance  personnel  for  OJT.  Rate  the 
adequacy  of  contractor-furnished  theory  of  operation 
training  to  prepare  maintenance  personnel  for  OJT. 

LTM  64  -  Knowledge  Training  (Type  4) 

Knowledge  Training  (Type  4)  refers  to  training 
provided  by  AETC  field  training  detachments  and  is 
similar  to  Type  1  training  in  that  it  provides  general 
task  knowledge.  This  is  designed  to  evaluate  the 
adequacy  of  the  theory  of  operation  training  to 
prepare  maintenance  personnel  for  OJT. 
Considerations  for  evaluating  Type  4  training  are 
identical  to  those  used  to  evaluate  Type  1  training. 
Rate  the  adequacy  of  field  training  detachment 
furnished  theory  of  operation  training  to  prepare 
maintenance  personnel  for  OJT. 

LTM  65  -  Proficiency  Training 

Proficiency  Training  refers  to  training  which  teaches 
the  hands-on  skills  necessary  to  accomplish  portions 
of  routine  and  critical  tasks.  Proficiency  training  is 
achieved  through  OJT.  This  is  designed  to  evaluate 
the  adequacy  of  task-applicable  OJT  received 
(contractor,  LTD,  cross-utilization  training  [CUT], 
and  system  upgrade  training  as  a  result  of  specific 
system  modifications  or  enhancements).  Rate  the 
adequacy  of  task-applicable  OJT  training  received 
(contractor,  LTD,  CUT,  and  system  upgrade). 
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8.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

8.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 


analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

8.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single-rating  conclusion  for  the 
ILS  element  based  on  mission  impact. 

The  TTS  element  directly  impacts  MP  and  DI  and 
will  be  taken  into  consideration  when  making  the 
overall  assessments  of  these  elements. 
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9.0  COMPUTER  RESOURCES  SUPPORT  (CRS) 


9.1  Description 

Computer  resources  support  (CRS)  is  the  hardware, 
software,  personnel  and  facilities  needed  to  operate 
and  support  an  aircraft  system.  Computer  resources 
support  includes  all  designated  computer  hardware, 
software,  and  firmware  embedded  into  or  in  support 
of  the  weapon  system.  Computer  Resources  Support 
also  includes  all  documentation  necessary  to  support 
and  operate  computer  hardware,  software,  and 
firmware. 

Computer  hardware  refers  to  the  on-aircraft 
computers  (i.e.,  LRUs)  and  microprocessors  (i.e., 
integrated  circuits  [ICs]),  single  chip  minicomputers 
(i.e.,  laptop  computers),  mainframe  computers  and 
peripherals  (i.e.,  LAN  networks,  disk  drivers). 
Computer  software  refers  to  the  set  of  instructions 
and  data  necessary  for  the  hardware  to  perform  its 
required  functions.  Computer  firmware  is  an 
integrated  circuit  made  for  a  hardware  unit  and 
software  or  data  integrated  from  a  functional  entity. 

9.2  Objectives 

The  CRS  element  evaluates  the  operational  and 
support  capabilities  of  the  various  computer  resource 
systems.  The  evaluations  will  center  on  the  overall 
operational  and  support  concepts  of  computer 
resources,  the  effectiveness  of  computer  resources, 
quality  and  accuracy  of  data,  and  man/machine 
interface.  The  CRS  evaluation  will  include 
evaluation  of  the  built-in-test  (BIT)  system,  all 
computer  resources  that  interface  with  the  test 
article,  all  off-equipment  computer  resources  the 
maintainer  may  interface  with  to  complete  a  task, 
and  all  test  article  systems  involving  computer 
resources. 

9.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  CRS 
element: 

Reliability  &  Maintainability 

LTM  70  -  CRS  Reliability 

LTM  71  -  CRS  Maintainability 

LTM  72  -  Diagnostic  System  Adequacy 

Human  Factors 

LTM  73  -  Ease  of  Use 

Logistics 

LTM  74  -  Functional  Utility 
LTM  75  -  Resource  Effectiveness 

9.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  CRS  element  and  are  addressed  in 
detail  in  section  9.4.1: 


CRS  Reliability 
CRS  Maintainability 
Diagnostics  System  Adequacy 

9.4.1  Evalnation  Measures 

What  will  be  evaluated  under  this  element  is  the  BIT 
system.  In  this  element,  the  evaluation  will 
primarily  focus  on  the  2-digit  level  WUCs  associated 
with  computer  resources.  Evaluation  to  the  3,  4  or  5 
WUC  levels  will  be  optional. 

Additionally,  the  concept  for  evaluating  the 
computer  resource’s  ability  to  operate  and  support 
the  system  will  be  equated  to  the  levels  of  R&M 
specified  and  measured  for  that  specific  computer 
resource.  Simply  put,  the  R&M  characteristics  of  the 
computer  resources  and  adequacy  of  the  BIT  system 
greatly  influences  the  overall  system’s  support, 
operation,  and  effectiveness.  The  assumption  is 
made  here  that  reliability  and  maintainability  are 
inherent  characteristics  of  computer  resources  and 
are  essential  in  achieving  overall  mission  and  combat 
effectiveness,  as  well  as  achieving  other 
characteristics  such  as  speed,  and  navigation 
accuracy.  A  computer  system  that  has  poor  R&M  is 
not  capable  of  fully  operating  and  supporting  the 
weapon  system.  For  example,  an  aircraft  is  partially 
operating  if  a  number  of  the  computer  resources  (i.e., 
mission  computers,  electronic  flight  control  system 
[EFCS])  are  always  being  fixed  (maintainability)  or 
awaiting  parts  (reliability).  The  more  reliable  and 
maintainable  a  computer  resource  is,  the  better  it 
operates  and  supports  the  weapon  system. 

Also,  in  this  DAP  the  assumption  is  made  that  the 
functionality  aspects  of  a  computer  resource  are 
directly  proportional  to  the  inherent  characteristics  of 
their  own  built-in  R&M  attributes.  That  is,  the 
electronic  circuitry  of  a  computer  resource  can  not 
satisfactorily  perform  its  designated  functions  (i.e., 
providing  accurate  measurements)  unless  it  is 
designed  with  highly  reliable  electronic  components. 
For  example,  an  shop  replacement  unit  (SRU)  may 
be  partially  performing  its  functions  (i.e.,  providing 
signals  to  another  SRU)  because  it  has  poor  reliable 
components,  which  equates  to  poor  aircraft  operation 
and  support. 

To  support  the  evaluation,  measured  CRS  R&M  and 
ILS  information  will  be  used  to  help  determine  if  the 
system  is  achieving  its  intended  supportability, 
operations,  and  objectives.  In  this  evaluation,  CRS 
R&M  and  ILS  characteristics  will  be  translated  into 
explicit  supportability-related  parameters  that  govern 
the  operation  and  effectiveness  of  the  aircraft  system 
and  each  of  its  accompanying  components. 


34 


LTM  70  -  CRS  Reliability 

From  an  R&M  standpoint,  computer  resources’ 
reliability  determines  the  aircraft  system’s  ability  to 
operate  as  planned.  For  the  CRS  element,  the 
reliability  of  the  aggregate  or  individual  computer 
resources  will  be  the  baseline  for  which 
supportability  will  be  defined.  Under  this  element, 
the  reliability  parameters  for  the  aggregate  or 
individual  CRS’  systems  will  be  evaluated  and 
translated  into  supportability  assessments  using  the 
following  calculations: 


Unconfirmed  Faults 

FA=  - 

X  1 00  T otal  Number  of  Faults  Reported 


Mean  Time  Between  Maintenance  (Inherent). 
MTBM(l)  is  described  in  the  section  3.4.1. 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  in  section  3.4.1. 

Mean  Time  Between  Removals  (MTBR).  MTBR  is 
described  in  section  3.4.1. 

LTM  71  -  CRS  Maintainability 

Maintainability  influences  mission  effectiveness  and 
readiness.  Mission  effectiveness  and  readiness  can 
be  equated,  respectively,  to  the  maintenance  man¬ 
hour  per  flying  hours  (MMH/FH)  and  the  mean  man¬ 
hours  to  repair/restore  (MMTR)  functions  during 
missions.  MMFI/FH  and  MMTR  summaries  of  the 
CRS  will  be  analyzed  as  follows: 


Measures  of  false  alarms  (FA).  False  alarms  are 
faults,  where  upon  investigation,  it  is  found  the  fault 
cannot  be  confirmed.  May  be  expressed  as  a  total 
number,  percentage,  rate  of  occurrence,  probability 
of  occurrence,  etc. 

Formula  7:  (Percentage) 


Number  of  Correct  Detections 

Pcd=  - 

X  1 00  Total  Number  of  Confirmed  Faults 


Percent  of  correct  detection  (Pcd)  given  that  a  fault 
has  occurred:  The  number  of  correct  detections 
divided  by  the  total  number  of  confirmed  faults  times 
100  (to  express  the  quotient  as  a  percent). 

Formula  8:  (Percentage) 


Number  of  Correct  Fault  Isolations 

Pcfi=  -  X  100 

Number  of  Correct  Detections 


Percent  of  correct  fault  isolation  (Pcfi)  (and  correct 
fault  location)  given  correct  detection:  The  number 
of  correct  fault  isolations  (and/or  correct  fault 
locations)  divided  by  the  number  of  correct 
detections 

Formula  9:  (Percentage) 


Total  Time  Reqd  to  Locate  Faults 

MTTFL=  . 

Total  Number  of  Faults 


Direct  Maintenance  Man-Hours  per  Flight  Hour. 
DMMH/FH  is  described  in  section  3.4.1. 

LTM  72  -  Diagnostics  System  Adequacy 

An  analysis  of  the  BIT  system  adequacy  will  be 
performed  to  determine  the  maintainability 
capabilities  of  the  aircraft.  Built-in-test  usually 
serves  two  functions:  (1)  to  monitor  and  report 
system  status  to  an  operator;  and  (2)  to  be  used  as  a 
maintenance  tool  for  system  diagnostics  and  repairs. 
The  objective  of  the  assessment  will  be  to  determine 
the  adequacy  of  the  BIT  system  as  a  tool  for 
conducting  diagnostics  and  repairs.  That  is,  its 
capability  to  confirm  faults,  locate  and  isolate  the 
cause  of  the  fault,  and  perform  repairs.  The 
following  BIT  factors  of  measures  of  false  alarms 
(FA),  percent  of  correct  detection  given  a  fault  has 
occurred  (Pcd),  percent  of  correct  fault  isolation  (and 
correct  fault  location)  given  correct  detection  (Pcfi), 
and  mean  time  to  fault  locate  (MTTFL)  factors  will 
be  used  to  determine  and  assess  the  adequacy  (i.e., 
system  effectiveness)  of  the  BIT  system. 


Mean  Time  to  Fault  Locate  (MTTFL):  The  total 
amount  of  time  required  to  locate  faults  divided  by 
the  total  number  of  faults. 

Formula  10:  (Mean) 

Missing  formula?  ? 

9.4.2  Measurement  Criteria 

The  main  objective  of  this  evaluation  is  to  identify 
any  CRS  supportability  trends/ shortfalls  or  technical 
voids  which  will  significantly  impact  the  ability  of 
the  system  to  meet  its  specified  R&M  and  ILS 
requirements.  These  objectives  will  be  met  by 
verifying  attainment  of  the  technical  CRS  R&M  and 
ILS  performance  requirements  specified  and  defined 
in  the  aircraft  specifications  and  LSAR  database. 
Additionally,  application  of  the  formulas  established 
and  defined  for  the  other  nine  ILS  elements  under 
this  DAP  will  be  used  for  evaluation  purposes.  This 
element  will  also  look  closely  at  the  failure  rates 
experienced  during  testing  (i.e.,  DT&E). 
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9.4.3  Analysis  Techniques  and  Tools 

Applications  of  the  SEDS  and  LSAR  databases  in 
assisting  to  identify  R&M,  ILS  and  BIT 
design-related  shortfalls  and  targets  for  subsequent 
design  corrections  will  be  used  for  the  evaluation. 

9.5  Human  Factors 

The  following  LTM  influences  the  CRS  element  and 
is  addressed  in  section  9.5.1 : 

LTM  73  -  Ease  of  Use 

9.5.1  Evaluation  Measures 

Logistic  test  measure  73  will  be  evaluated  to 
determine  the  adequacy  of  the  BIT  system  to 
perform  maintenance  tasks  in  fault  diagnosis. 

LTM  73 -Ease  of  Use 

Ease  of  use  refers  to  the  ability  of  the  maintenance 
person  to  understand  and  effectively  use  the  BIT 
system  to  perform  maintenance  tasks  in  fault 
diagnosis.  This  LTM  is  designed  to  evaluate  the 
usefulness  of  the  BIT  system  as  a  fault  isolation  aid. 
Rate  the  adequacy  of  the  BIT  system  as  a  fault 
isolation  aid. 

9.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

9.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

9.6  Logistics 

The  following  LTMs  influence  the  CRS  element  and 
are  addressed  in  detail  in  section  9.6.1: 


LTM  74  -  Functional  Utility 
LTM  75  -  Resource  Effectiveness 

9.6.1  Evaluation  Measures 

Logistics  test  measures  74  and  75  will  be  evaluated 
to  determine  the  adequacy  and  overall  operational 
and  support  concepts  of  the  computer  resources  and 
the  effectiveness  of  the  computer  resources. 

LTM  74  -  Functional  Utility 

Functional  utility  refers  to  the  capability  of  a 
particular  system  to  function  as  per  design.  This  is 
designed  to  evaluate  the  adequacy  of 
automated/initiated  diagnostic  systems  in  relation  to 
accurate  and  efficient  fault  detection,  reporting, 
isolation,  recording,  and  correlating  of  malfunctions. 
Rate  the  adequacy  of  air  vehicle  automated/initiated 
diagnostics  in  relation  to  accurate  and  efficient  fault 
detection,  reporting,  isolation,  recording,  and 
correlating  of  malfunctions. 

LTM  75  -  Resource  Effectiveness 

Resource  Effectiveness  refers  to  the  ability  of 
computer  resources  that  interface  with  the  test  article 
to  efficiently  operate  and  support  the  aircraft  system. 
This  is  designed  to  evaluate  the  computer  resources 
that  interface  with  the  test  article  for  effectiveness  in 
uploading  and  downloading  data  and  information 
from  various  systems,  both  on-  and  off-equipment. 
Rate  the  adequacy  of  the  hardware  and  software 
utilized  upload  and  download  data  and  information 
from  various  systems,  both  on-  and  off-equipment. 

9.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

9.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 
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9.7  Overall  Element  Analysis  Ths  CRS  element  directly  impacts  the  MP  and  DI 

elements,  and  will  be  taken  into  consideration  when 

Overall  element  analysis  will  be  a  cumulative  effort  making  the  overall  assessment  of  these  elements. 

by  all  three  disciplines.  Individual  results  and 

conclusions  reached  by  each  discipline  will  be 

combined  to  form  a  single-rating  conclusion  for  the 

ILS  element  based  on  mission  impact. 
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10.0  FACILITIES  (FA) 

10.1  Description 

Facilities  (FA)  consists  of  all  permanent  and 
semipermanent  structures  required  to  support  the 
weapon  system.  The  aircraft  facility  requirements 
have  been  determined  through  the  logistics  support 
analysis  (LSA)  process  using  the  operational  and 
maintenance  needs  of  the  weapons  system  as 
baseline.  Examples  of  aircraft  facility  requirements 
needing  evaluation  include  maintenance  hangars, 
fuel  pits,  engine  run  pads,  wash  racks,  parking 
ramps,  and  fuel  cells. 

These  FAs  must  be  able  to  support  the  operational 
and  support  concepts  of  the  weapon  system  as 
identified  in  the  Integrated  Logistics  Support  Plan 
(ILSP). 

10.2  Objective 

The  objective  of  the  FA  evaluation  is  to  determine  if 
the  planned  FAs  adequately  support  the  needs  of  the 
weapon  system.  The  evaluation  will  identify  facility 
shortcomings  and  possible  corrective  action  and  the 
need  for  additional  or  alternative  facilities. 

10.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  FA 
element: 

Reliability  and  Maintainability 

LTM  80  -  Reliability 
LTM  81  -  Maintainability 

Human  Factors 

LTM  82  -  Safety 

Logistics  Test 

LTM  83  -  Utilities 
LTM  84  -  Capacity 

10.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  FA  element  and  are  addressed  in  detail 
in  section  10.4.1. 

LTM  80  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

Mean  Time  Between  Removal  —  MTBR 


LTM  81  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour  — 
DMMH/FH 

Mean  Man-Hours  to  Repair  -  MTTR 

10.4.1  Evaluation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  the  LTMs  cited  in  section  10.3  to  FA. 

LTM  80  -  Reliability 

The  aircraft  system  reliability  will  be  evaluated 
under  this  LTM.  This  LTM  evaluates  the  optimal 
reliability  performance  characteristics  of  the  aircraft 
system.  The  objective  is  to  determine  if  the  planned 
reliability  and  ILS  criteria  have  been  met. 

Mean  Time  Between  Maintenance  (Inherent). 
MTBM(I)  is  described  in  the  section  3.4.1. 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  in  section  3.4.1. 

Mean  Time  Between  Removal.  MTBR  is  described 
in  seetion  3.4.1. 

LTM  81  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour. 
DMMH/FH  is  described  in  section  3.4.1. 

Mean  Time  to  Repair.  MTTR  is  described  in  the 
Maintenance  Planning  section  3.4.1. 

10.4.2  Measurement  Criteria 

Maintenance  tasks  which  require  a  facility  to 
properly  perform  the  maintenance  will  be  included  in 
this  evaluation.  Reliability  and  maintainability 
metrics  will  aid  in  determining  if  the  proper  number 
of  facilities  will  be  available  to  perform  the  required 
amount  of  maintenance. 

10.4.3  Analysis  Techniques  and  Tools 

Actual  values  for  the  R&M  metrics  will  be  obtained 
from  SEDS. 

10.5  Human  Factors 

The  following  LTM  influence  the  Facilities  element 
and  is  addressed  in  section  10.5.1. 

LTM  82  -  Safety 
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10.5.1  Evaluation  Measures 

LTM  82  -  Safety 

Safety  refers  to  all  safety-related  factors  designed 
into  facilities  to  ensure  a  safe  and  healthful  work  area 
for  personnel.  This  LTM  is  designed  to  evaluate  the 
adequacy  of  facilities  in  relation  to  fire  protection 
equipment,  ease  of  exit  and  entry,  sufficient 
ventilation,  sufficient  emergency  equipment 
(eyewashes,  first  aid  kits),  etc.  Rate  the  adequacy  of 
the  facilities  safety  provisions  considering  fire 
protection  equipment,  ease  of  exit  and  entry, 
sufficient  ventilation,  sufficient  emergency 
equipment  (eyewashes,  first  aid  kits),  etc. 

10.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when  rated  5 
or  higher  on  the  LTDS  and  there  are  no  significant 
negative  comments  noted.  Logistics  test  measures  rated 
4  or  below,  or  having  significant  negative  comments, 
will  be  analyzed  in  accordance  with  the  procedures  in 
Analysis  Techniques  and  Tools. 

10.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  eomments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  eomments  applieable  to 
a  production-representative  eondition  will  be 
analyzed  to  determine  root  cause  of  the  defieieney. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionies, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

10.6  Logistics  Test 

The  following  LTMs  influence  the  FA  element  and 
are  addressed  in  detail  in  section  10.6.1 : 

LTM  83  -  Utilities 
LTM  84  -  Capacity 

10.6.1  Evaluation  Measures 

Logistics  test  measures  83  and  84  will  be  evaluated 
to  determine  if  the  planned  facilities  adequately 
support  the  needs  of  the  weapon  system. 

LTM  83  -  Utilities 

Utilities  refers  to  the  test  site  utilities  (i.e.,  water, 
high  pressure  air,  temperature  and  humidity  control. 


lighting,  electrical,  etc.)  and  their  ability  to  support 
the  air  vehicle  and  SE.  This  is  designed  to  evaluate 
the  adequacy  of  site  utilities  and  identify 
shortcomings  that  could  indicate  additional  planned 
facilities  requirements.  Rate  the  ability  of  the  site 
utilities  (high  pressure  air,  temperature  and  humidity 
control,  lighting,  electrical,  etc.)  to  support  the  air 
vehicle  and  SE. 

LTM  84  -  Capacity 

Capacity  refers  to  the  test  site  facility  size.  This  is 
designed  to  evaluate  the  adequacy  of  the  test  site  facility 
size  and  layout  considering  ease  of  aircraft  movement, 
ability  to  utilize  SE  around  the  aircraft,  and  equipment 
storage  space  (including  both  SE  and  parts).  Rate  the 
adequacy  of  the  test  site  facility  size  and  layout 
considering  ease  of  aircraft  movement,  ability  to  utilize 
SE  around  the  aircraft,  and  equipment  storage  space 
(including  both  SE  and  parts). 

10.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when  rated  5 
or  higher  on  the  LTDS  and  there  are  no  significant 
negative  comments  noted.  Logistics  test  measures  rated 
4  or  below,  or  having  significant  negative  comments 
will  be  analyzed  in  accordance  with  the  procedures  in 
Analysis  Techniques  and  Tools. 

10.6.3  Analysis  Techniques  and  Tools 

Logisties  test  measures  rated  4  or  below,  or  having 
signifieant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

10.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single-rating  conclusion  for  the 
ILS  element  based  on  mission  impact. 

In  order  to  make  an  accurate  assessment  of  the  FA 
element,  the  impacts  on  MP  and  DI  elements  has  to 
be  taken  into  consideration. 
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11.0  PACKAGING,  HANDLING,  STORAGE,  AND  TRANSPORTATION  (PHS&T) 


11.1  Description 

Packaging,  handling,  storage,  and  transportation 
(PHS&T)  encompasses  the  requirements,  resources, 
processes,  procedures,  design  considerations,  and 
methods  to  ensure  that  all  system  equipment  and 
support  items  are  preserved,  packaged,  handled, 
stored,  and  transported  properly.  This  element 
includes  all  special  provisions,  containers  (reusable 
and  disposable),  and  supplies  necessary  to  support 
packaging,  preservation,  storage,  handling,  and/or 
transportation  of  prime  mission  equipment,  test  and 
support  equipment,  spares  and  repair  parts, 
personnel,  technical  data,  and  mobile  facilities.  In 
essence,  this  element  basically  covers  the  initial 
distribution  of  products  and  the  transportation  of 
personnel  and  materials  for  maintenance  purposes. 

11.2  Objective 

The  purpose  of  this  evaluation  is  to  ensure  that 
PHS&T  effectively  supports  the  aircraft.  The 
evaluation  will  determine  the  capability  of  the 
PHS&T  equipment  to  support  maintenance 
(including  suitability,  safety,  and  ease  of  use); 
determine  if  the  procedures  used  comply  with  the 
best  commercial  standards;  identify  PHS&T-related 
deficiencies  and  possible  corrective  actions;  and 
identify  additional  or  alternative  PHS&T 
requirements. 

The  PHS&T  evaluations  will  be  accomplished 
during  Climatic  Lab  and  All-Weather  deployments 
because  all  shipments  for  these  activities  are  required 
to  comply  with  applicable  MIL-STDs. 

11.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the 
PHS&T  element: 

Reliability  &  Maintainability 

LTM  90  -  Relaibility 

Human  Factors 

LTM  91  -  Safety 

Logistics 

LTM  92  -  Suitability 

11.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  PHS&T  element  and  are  addressed  in 
detail  in  section  1 1 .4. 1 . 


LTM  90  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 
-  MTBM(C) 

Mean  Time  Between  Removal  —  MTBR 

11.4.1  Evalnation  Measures 

The  following  R&M  characteristics  and  factors, 
which  are  quantitative  in  nature,  provide  a  basis  for 
relating  the  LTMs  cited  in  section  11.3  to  PHS&T. 

LTM  90  -  Reliability 

The  aircraft  system  reliability  will  be  evaluated 
under  this  LTM.  This  LTM  evaluates  the  optimal 
reliability  performance  characteristics  of  the  aircraft 
system.  The  objective  is  to  determine  if  the  planned 
reliability  and  ILS  criteria  have  been  met. 

Mean  Time  Between  Maintenance  (Inherent). 
MTBM(I)  is  described  in  section  3.4.1 . 

Mean  Time  Between  Maintenance  (Total 
Corrective).  MTBM(C)  is  described  section  3.4.1. 

Mean  Time  Between  Removal.  MTBR  is  described 
in  section  3.4.1. 

11.4.2  Measurement  Criteria 

Maintenance  tasks,  which  require  PHS&T  to 
properly  perform  the  maintenance,  will  be  included 
in  this  evaluation.  Reliability  and  maintainability 
metrics  will  aid  in  determining  if  the  proper  PHS&T 
is  being  perform  adequately  to  support  required 
maintenance. 

11.4.3  Analysis  Techniques  and  Tools 

Actual  values  for  the  R&M  metrics  will  be  obtained 
from  SEDS. 

11.5  Human  Factors 

The  following  LTM  influence  the  PHS&T  element 
and  is  addressed  in  detail  in  section  1 1 .5. 1 . 

LTM  91  -  Safety 
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11.5.1  Evaluation  Measures 

LTM  91  -  Safety 

Safety  refers  to  the  PHS&T  procedures  which  ensure 
minimal  risk  to  personnel  and  equipment.  This  was 
designed  to  evaluate  the  adequacy  of  safeguards  for 
PHS&T  of  equipment,  parts,  and  hazardous 

materials.  Rate  the  adequacy  of  safeguards  for 
PHS&T  of  equipment,  parts,  and  hazardous 

materials. 

11.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

11.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  defieieney. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

11.6  Logistics 

The  following  LTM  influences  the  PHS&T  element 
and  is  addressed  in  section  1 1 .6. 1 . 


11.6.1  Evaluation  Measures 

LTM  92  -  Suitability 

Suitability  refers  to  the  appropriateness  of  the 
PHS&T  of  parts  and  equipment.  This  is  designed  to 
evaluate  the  adequacy  of  packaging  materials  and 
procedures  to  prevent  deterioration  or  damage  of 
parts  and/or  equipment  during  shipment  and  storage. 
Rate  the  adequacy  of  packaging  materials  and 
procedures  to  prevent  damage  to  parts  and  equipment 
during  shipment  and  storage. 

11.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

11.6.3  Analysis  Techniques  and  Tools 

Logisties  test  measures  rated  4  or  below,  or  having 
signifieant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
exeluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  eleetrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 

11.7  Overall  Element  Analysis 

The  PHS&T  element  directly  impacts  MP  and  DI 
and  will  be  taken  into  consideration  when  making 
the  overall  assessments  of  these  elements. 


LTM  92  -  Suitability 
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12.0  DESIGN  INTERFACE  (DI) 

12.1  Description 

Design  interface  (DI)  encompasses  the  techniques, 
processes,  and  procedures  undertaken  to  ensure  that 
air  vehicle  supportability’  is  addressed  in  the  systems 
engineering  process.  In  other  words,  the  DI  element 
monitors  the  weapon  system  design  process  to  ensure 
that  logistic -related  design  parameters  are  fully 
integrated  into  the  weapon  system  and  equipment 
design.  The  DI  element  is  unique  in  that  it 
cross-references  to  the  other  nine  ILS  elements.  That 
is,  SS,  SE,  and  the  other  ILS  elements  are  input 
functions  to  the  DI  element.  The  DI  element  is  also 
unique  in  that  it  specifically  integrates  the  following 
logistic -related  design  parameters  to  the  aircraft 
supportability,  operational  performance  and  support 
cost  requirements: 

•  Reliability  and  Maintainability  (R&M) 

•  Human  Factors  (HSI) 

•  Energy 

•  Survivability 

•  Standardization  and  Interoperability 

•  Test  and  Evaluation  (T&E) 

For  example,  the  R&M  design  parameters  relate  to 
the  DI  element  in  that  these  characteristics  influence 
the  supportability  requirements  (system  design 
characteristics  and  planned  logistic  resources), 
operational  performance  (mission  effectiveness, 
system  availability)  and  economics  (life-cycle  cost) 
of  the  weapon  system.  Survivability  relates  to  the 
DI  element  in  that  it  addresses  those  logistics-related 
design  parameters  required  to  preserve  survivability 
of  the  weapon  system.  The  energy  subelement 
addresses  the  design  characteristics  of  the  aircraft 
system  that  cause  it  to  require  and  consume  critical 
energy  resources,  such  as  petroleum  and  fuels,  liquid 
oxygen,  hydraulic  fluids,  etc.  Interoperability  and 
standardization  relates  to  the  DI  element  in  that  it 
addresses  the  systems  ability  to  interface,  operate, 
and  be  supported  by  inventory  or  off-the-shelf 
standard  equipment  and  spare  parts. 

12.2  Objective 

The  objective  of  the  DI  element  is  to  determine  if  the 
overall  weapon  system  and  its  associated  equipment 
operates,  interfaces,  and  is  supported  by  inventory  or 
off-the-shelf  standard  equipment  and  spare  parts. 

The  ultimate  objective  is  to  assess  that  all  aircraft 
system  supportability  requirements  provide  an 
opportunity  for  reducing  logistics  support  costs 


Supportability  is  the  degree  to  which  system  design 
characteristics  and  planned  logistic  resources  meet  system 
peacetime  readiness  and  wartime  utilization  requirements. 


and/or  enhance  system  readiness.  In  the  context  of 
this  DAP,  the  following  criteria  apply. 

Criteria  1.  Aircraft  system  supportability 
evaluations  will  be  related  only  to  the 
logistics-related  design  parameters  of  R&M,  Human 
Factors,  and  Logistics  testing  with  the  help  of  the 
other  nine  ILS  elements.  The  evaluations  will 
attempt  to  identify  any  design  requirement  related 
deficiencies  and/or  possible  corrective  actions.  Any 
design  requirements  related  deficiencies  will  be 
assessed  for  impact  on  logistic  support  resource 
requirements. 

Criteria  2:  In  line  with  the  DT&E  objectives,  the 
analyses  provided  here  will  focus  only  on  evaluating 
the  system  supportability  design  requirements  (i.e., 
attainment  of  technical  performance  objectives  such  as 
system  design  characteristics  and  planned  logistic 
resources).  Operational  Test  &  Evaluation  (OT&E)  of 
the  system's  operational  effectiveness  objectives  (i.e., 
mission  effectiveness  and  system  suitability)  will  not 
be  evaluated  under  this  element. 

12.3  Logistics  Test  Measures  (LTMs) 

The  following  LTMs  will  be  evaluated  for  the  DI 
element: 

Reliability  &  Maintainability 

LTM  100  -  Reliability 
LTM  101  -  Maintainability 

Human  Factors 

LTM  1 02  -  Human/Machine  Interface 
LTM  103  -  Safety 

Logistics 

LTM  1 04  -  Machine/Machine  Interface 

LTM  105  -  Design 

LTM  1 06  -  Interoperability 

12.4  Reliability  and  Maintainability 

The  following  R&M  characteristics  and  factors 
influence  the  DI  element  and  are  addressed  in  detail 
in  section  12.4.1. 

LTM  too  -  Reliability 

Mean  Time  Between  Maintenance  (Inherent)  — 
MTBM(I) 

Mean  Time  Between  Maintenance  (Total  Corrective) 

-  MTBM(C) 

Mean  Time  Between  Removal  —  MTBR 

LTM  101  -  Maintainability 

Direct  Maintenance  Man-Hours  per  Flight  Hour  — 
DMMH/FH 

Mean  Man-Hours  to  Repair.—  MMTR 
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12.4.1  Evaluation  Measures 

The  aircraft  system's  supportability  and  operational 
effectiveness  (i.e.,  suitability  and  readiness 
objectives)  are  functions  and  determinants  of  the 
system's  R&M  characteristics  and  ILS  elements. 
Reliability  and  maintainability  and  ILS 
characteristics  influence  and  interact  with  the  system 
engineering  process.  Accordingly,  collected  R&M 
and  ILS  information  can  help  determine  if  the 
logistic  support  requirements  are  being  reduced  and 
attained.  In  this  DAP,  R&M  and  ILS  objectives  will 
be  translated  into  explicit  supportability-related 
design  parameters  that  govern  the  design  of  the 
system  and  each  of  its  accompanying  components, 
and  into  logistic  support  parameters  than  can  help 
determine  the  development,  test  and  evaluation,  and 
health  of  the  aircraft  system. 

LTM  100  -  Reliability 

From  a  logistics  standpoint,  reliability  is  a  measure 
of  the  aircraft  system's  ability  to  operate  as  planned 
under  the  defined  operational  and  support  concept 
using  specified  resources.  Logistics  reliability  is  the 
baseline  from  which  supportability  is  defined.  It  is 
through  logistics  reliability  that  we  determine  what 
will  fail  and  at  what  intervals  it  will  fail. 

Reliahility  Analysis  Summary  (Hardware).  A  list  of 
failure  modes  that  have  the  greatest  impact  upon  the 
system's  hardware  reliability  will  be  compiled.  The 
information  provided  in  the  list  will  be  the  basis  for 
identifying  problem  areas  needing  emphasis,  changes 
or  modifications.  The  following  list  of  problem 
areas,  used  for  the  reliability  analysis,  will  be 
extracted  from  the  following  SEDS  and  LSAR  data 
products,  to  include  the  material  improvement 
program  (MIP)  and  service  reports  (SR)  data 
products: 


SEDS 

■  Top  25  problem 
areas 

■  AMTS  (Active 
Manhour  Task 
Summary) 

■  CMP  (Component 
Discrepancy 
Report)  Analysis 
(FMECA) 

■  Reliability  Failure 
Summaries 

Note:  Detailed  description  of  the  SEDS  products 
listed  above  will  not  be  provided  here.  These  are 
described  and  defined  in  detail  under  the  SEDS 
user's  manuals.  The  SEDS  user's  manuals  can  be 
obtained  through  the  engineering  R&M  section. 
Equally,  the  LSAR  MIL-STD  1338  1A/2A/2B 
(Reference  6)  addresses  in  more  detail  the  RCM  and 
FMECA  processes  and  how  they  are  established. 
The  RCM  and  FMECA  data  products/results  are 
obtained  from  the  LSAR-050  and  LSAR-055 
products  contained  within  the  aircraft  LSAR 
database. 

Reliability  Analysis  Summary  (Software).  This  factor 
will  be  used  to  measure  the  software  reliability 
component  of  the  total  aircraft  system's  reliability 
(i.e.,  hardware  plus  software  reliability).  Under  this 
element,  only  OFP/ flight  software  related  reliability 
will  be  evaluated.  Software-induced  failures,  as 
defined  within  the  context  of  this  paper,  refer  to 
those  induced  failures  that  result  from  the 
combination  of  the  environment  and  design  faults 
within  the  software.  An  environmental  failure  in 
software  occurs  when  environmental  conditions  (i.e., 
altitude  flying,  maintenance  actions,  etc.)  create  a  set 
of  inputs  causing  faults  within  the  sections  of 
software  code  to  be  executed.  A  software  design 
failure  is  a  result  of  a  set  of  inputs  which  the 
designers/programmers  did  not  account  for  properly. 

Software  reliability  analysis  will  use  the  software 
failures  reported  in  the  SEDS  maintenance  actions 
but  not  necessarily  reported  via  the  MIPs  or  SRs. 
These  data  are  recorded  in  SEDS  workfields  <4B 
through  5B>.  All  data  is  severity  coded  1  through  5 
in  SEDS,  but  only  codes  1  and  2  are  used  for 
performing  reliability  calculations. 


LSAR 

Reliability  Centered 
Maintenance  (RCM) 

Failure  Mode 
Effects  and 
Criticality 
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Mean  Time  between  Maintenance  (T otal  Corrective). 
Mission  effectiveness  is  also  influenced  by 
reliability.  Mission  effectiveness  can  be  described  in 
terms  of  mean  time  between  maintenance  (MTBM). 
The  MTBM(C)  is  the  mean  time  between 
unscheduled,  on-equipment  corrective  maintenance 
actions,  with  time  expressed  as  aircraft  flying  hours. 
Where  total  corrective  maintenance  actions  is  the 
sum  total  of  the  number  of  Type  1  (inherent),  Type  2 
(induced)  and  Type  6  (no  defect),  CND,  or  FOM 
maintenance  actions.  Detailed  descriptions  and 
definitions  of  these  types  of  maintenance  actions  are 
found  under  the  SEDS  Data  Collection  Procedure 
Guide.  The  MTBM(C)  summaries  will  also  be 
provided  under  this  report  to  supplement  the  above 
hardware  reliability  analysis.  MTBM(C)  includes; 

•  corrective  maintenance  actions  (sum  of 
inherent,  induced,  and  no  defect) 

•  unscheduled  maintenance 

•  time  in  flying  hours  (production  aircraft) 

•  on  equipment. 

Formula  1 : 


Total  Flight  Hours 

MTBM(C)  = . 

Total  Corrective  Maintenance  Aetions 


where; 


MTBR  excludes; 

•  removals  to  facilitate  other  maintenance 

•  TCTOs 

•  scheduled  maintenance 

•  nonrepairable  components. 

Formula  2; 


Total  Flight  Hours 

MTBR  = - 

Total  Number  of  Removals 


where; 

MTBR  is  the  mean  time  between  removals.  Where 
MTBR  indicates  the  number  of  equipment  removed 
from  the  aircraft  throughout  a  specified  number  of 
flying  hours.  Flight  hours  are  the  number  of  flying 
hours  for  a  specified  production  aircraft.  The  hours 
are  aircraft  flight  hours  for  aircraft  equipment  or 
equipment  operating  hours.  Equipment  removals  are 
the  number  of  equipment  removed  from  the  supply 
system  to  install  on  the  aircraft  during  the  specified 
flight  hours.  Total  flight  hours  can  be  obtained  from 
the  CTF  maintenance  debrief  Form  207  and  R&M 
cyclic  database.  Total  removals  or  retrieval 
maintenance  actions  can  be  derived  from  workfield 
<1>  of  SEDS  failure  reports. 


MTBM(C)  is  the  mean  time  between  maintenance 
with  total  correetive  actions.  Total  flying  hours  can 
be  obtained  from  CTF  maintenance  debrief  Form 
207  and  R&M  cyclic  database.  Total  corrective 
maintenance  actions  are  derived  from  SEDS  failure 
reports,  which  include  the  sum  of  the  inherent, 
induced  and  no  defects. 

Mean  Time  Between  Removal/Restore  (MTBR). 
Mission  readiness  and  supportability  is  also 
influenced  by  reliability.  Mission  readiness  and 
supportability  can  be  described  in  terms  of  MTBR. 
The  MTBR  measures  the  mean  time  interval  between 
removal  of  repairable  components,  with  time 
expressed  as  aircraft  flying  hours.  The  MTBR 
reflects  the  removal  or  restoration  readiness  of 
repairable  and  non-repairable  equipment  from  the 
weapon  system.  The  MTBR,  thus,  reflects  the 
removal  and  restoration  rate  (i.e.,  readiness).  The 
MTBR  is  the  mean  time  interval  between  removal  of 
any  repairable  or  nonrepairable  equipment  for 
indicated  or  actual  item  or  equipment  failures. 
MTBR  includes; 

•  production  aircraft  flying  hours 

•  indicated  failures  (RTOK) 

•  chargeable  removals; 

-  on  equipment 

-  repairable. 


LTM  101  -  Maintainability 

Maintenance  Analysis  Summary.  Maintenance 
analysis  summaries  will  consist  of  a  series  of 
analyses  detailing  maintenance  actions  such  as 
repairs,  inspections,  or  otherwise  maintenance  tasks 
that  support  the  weapon  system.  This  analysis  will 
help  identify  areas  of  high  maintenance  actions.  The 
summaries  will  utilize  the  inputs  recorded  in  the 
SEDS,  LSAR,  MIPS,  and  SRs  records.  The 
following  parameters  will  be  analyzed,  as  a 
minimum,  and  will  be  used  to  generate  information 
for  performing  the  analysis; 

•  task  descriptions  and  identification  of 

functions,  interval,  and  levels  of  repair 

•  task  frequencies 

•  task  resources  identification  by; 

-  spares  and  repair  parts 

-  skill  specialties 

-  man-hours  and  elapsed  times. 

Mission  Supportability.  Mission  supportability  is 
influenced  by  maintainability.  Mission 

supportability  is  also  influenced  by  reliability,  that  is, 
the  MTBR  functions  during  a  mission.  Mission 
supportability,  in  terms  of  maintainability,  is 
measured  in  elapsed  repair  times  (ET),  which 
considers  the  repair  times  in  hours  involved  in 
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aircraft  system  repairs.  Elapsed  repair  times  can  be 
determined  by  summing  the  total  repair  hours  in 
performing  each  Type  1,  Type  2  or  Type  6 
maintenance  tasks.  Use  formula  3  below. 

Formula  3: 


ETt  =  ^ETi  (hours) 


where: 

ETt  is  the  sum  total  of  all  maintenance  hours 
accumulated  for  the  test  fleet  or  for  a  specific 
production  aircraft.  ETt  includes  the  summation  of 
all  hours  accumulated  in  the  performance  of  Type  1 , 
Type  2,  and  Type  6  maintenance  actions.  Where  ETi 
(in  hours)  is  the  total  maintenance  hours  used  in 
performing  Type  1,  Type  2,  and  Type  6  maintenance 
actions. 

Critical  Maintenance  Analysis  Summary.  A  list 
identifying  critical  support  and  supportability-related 
design  factors  or  failure  modes  will  also  be  provided. 
The  list  will  provide  a  summary  of  maintenance 
tasks  that  exceed  specified  threshold  levels  such  as 
frequency,  elapsed  time  or  annual  maintenance  man¬ 
hours.  The  critical  maintenance  task  analysis  will 
break  each  maintenance  action  into  specific  subtasks 
in  order  to  identify  skill  requirements,  elapsed  times, 
task  frequencies,  personnel  required  at  each 
maintenance  level,  and  the  character  of  the  repair 
actions  (inspections,  scheduled,  unscheduled,  etc.). 
This  task  will  consist  of  detailed  analysis  of  the 
operation  and  maintenance  tasks  required  for 
supporting  the  aircraft  system.  The  summaries  will 
utilize  the  inputs  recorded  in  the  SEDS,  LSAR,  MIPs 
and  SRs.  The  specific  objectives  of  this  analytical 
activity  will  include  but  will  not  limited  to 
identifying: 

•  personnel  types  such  as  AFSC,  skill  levels, 
number  and  usage  factors 

•  elapsed  task  times  summaries 

•  man-hour  conversions  to  manpower 
requirements 

•  spares/repair  identification  to  support  the 
system 

•  total  flying  hours  (per  fleet,  per  production, 
etc.) 

•  unscheduled  maintenance  summaries 
(MMH/FH) 

•  scheduled  maintenance  summaries  (i.e., 
support  general,  inspections) 

•  average  maintenance  downtime  per  aircraft 
(hours) 

•  maximum  maintenance  downtime  per 
aircraft  (hours) 

•  maintenance  actions  by  WUC 


•  maintenance  frequencies  per  tasks  (i.e., 
monthly,  annual)  per  flight  hours 

•  maintenance  task  time  in  hours,  years,  etc. 

•  replacement  rates  for  systems/subsystems 

•  logistic  support  resources  used  for  each  task. 

Maintainability  Analysis  Summary.  An  analysis  of 
the  aircraft  BIT  system  adequacy  will  be  performed 
to  determine  the  maintainability  capabilities  of  the 
aircraft.  Built-in-test  usually  serves  two  functions: 
(1)  to  monitor  and  report  system  status  to  an 
operator;  and  (2)  to  be  used  as  a  maintenance  tool  for 
aircraft  system  diagnostics  and  repairs.  The 
objective  of  the  assessment  will  be  to  determine  the 
adequacy  of  the  aircraft  BIT  system  as  a  tool  for 
conducting  diagnostics  and  repairs.  That  is,  its 
capability  to  confirm  faults,  locate  and  isolate  the 
cause  of  the  fault,  and  perform  repairs.  Built-in-test 
factors  of  measures  of  FA,  Pcd  given  a  fault  has 
occurred,  Pcfi  (and  correct  fault  location)  given 
correct  detection,  and  MTTFL  will  be  used  to 
determine  and  assess  the  (system/subsy  stems) 
integrated  diagnostics  effectiveness. 

Percent  BIT  Accurate.  Percent  BIT  accurate  is 
described  in  the  Appendix  C. 

12.4.2  Measurement  Criteria 

The  main  objective  of  this  ILS  element,  as  it  pertains 
to  R&M,  is  to  verify  if  the  planned  aircraft  R&M 
allocations  are  met.  The  evaluation  and  analysis  will 
identify  any  R&M  trends/shortfalls  which 
significantly  impact  supportability  or  the  ability  of 
the  system  to  meet  the  specified  R&M  allocations. 
These  objectives  will  be  met  by  verifying  the 
attainment  of  the  technical  R&M 
requirements/specifications,  evaluation  of  the 
logistics  questionnaires  (LTDSs),  DRs,  and  PCRs 
effecting  the  maintenance  plan. 

12.4.3  Analysis  Techniques  and  Tools 

Applications  of  SEDS  and  LSAR  databases,  which 
assist  in  identifying  R&M  and  ILS  design-related 
shortfalls  and  targets  for  subsequent  design 
corrections,  will  be  used. 

12.5  Human  Factors 

The  following  LTMs  influence  the  DI  element  and 
are  addressed  in  detail  in  section  12.5.1. 

LTM  1 02  -  Human/Machine  Interface 
LTM  103  -  Safety 
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12.5.1  Evaluation  Measures 

Logistics  test  measures  102  and  103  will  be 
evaluated  to  determine  the  adequacy  of  the  air 
vehicle  design  for  minimizing  discomfort, 
distraction,  and  other  factors  which  degrade  human 
performance  or  increase  error. 

LTM  1 02  -  Human/Machine  Interface 

Human/Machine  Interface  refers  to  interaction  that 
takes  place  between  humans  and  the  air  vehicle 
during  operation  and  maintenance.  This  is  designed 
to  evaluate  the  air  vehicle  design  features  for 
unrestricted  physical  and  visual  access  to 
components  and  ease  of  access  to  high  maintenance 
areas.  Rate  the  adequacy  of  air  vehicle  design 
features  for  unrestricted  physical  and  visual  access  to 
components  and  ease  of  access  to  high  maintenance 
areas. 

LTM  103  -  Safety 

Safety  refers  to  the  ability  of  the  air  vehicle  design  to 
minimize  risk  of  personnel  injury  and  equipment 
damage  while  accomplishing  required  tasks  on  or 
around  the  aircraft.  This  is  designed  to  evaluate  the 
adequacy  of  design  features  to  minimize  risk  of 
personnel  injury  and  equipment  damage,  and  to 
identify  potentially  hazardous  conditions  requiring 
correction.  Rate  the  adequacy  of  the  design  features 
to  minimize  risk  of  personnel  injury  and  equipment 
damage  considering  the  conditions'  hazard  potential. 

12.5.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  test 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

12.5.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 


12.6  Logistics 

The  following  LTMs  influence  the  DI  element  and 
are  addressed  in  section  12.6.1: 

LTM  104  -  Machine/Machine  Interface 

LTM  105  -  Design 

LTM  1 06  -  Interoperability 

12.6.1  Evaluation  Measures 

Logistics  test  measures  104,  105,  and  106  will  be 
evaluated  to  determine  the  adequacy  of  the  air 
vehicle  design  characteristics  and  for  an  opportunity 
to  reduce  logistics  support  costs  and/or  enhance 
system  readiness. 

LTM  104  -  Machine/Machine  Interface 

Machine/Machine  Interface  refers  to  the  relative 
positioning  of  line  replaceable  units  (LRUs)  and  line 
replaceable  modules  (LRMs)  on  the  air  vehicle  and 
their  impact  on  unhindered  operation  and 
maintenance.  This  is  designed  to  evaluate  the 
operation,  removal,  and  replacement  of  LRUs/LRMs 
and  equipment  in  regard  to  interference  from  other 
LRUs/LRMs  (cables,  hydraulic  lines,  structures  or 
equipment  which  required  removal  to  facilitate  other 
maintenance).  Rate  the  adequacy  of  the  relative 
positioning  of  LRUs/LRMs  for  the  purpose  of 
unhindered  operation  and  maintenance. 

LTM  105  -  Design 

Design  refers  to  the  adequacy  of  system/component 
design  characteristics  in  relation  to  life-cycle  cost. 
This  is  designed  to  evaluate  the  adequacy  of  the 
overall  design  characteristics  of  the  air  vehicle  and  to 
identify  design  deficiencies  (potential  and 
confirmed)  that  could  lead  to  premature  failures  and 
increase  life  cycle  cost.  Examples  of  deficiencies  to 
be  identified  should  include  chaffing  conditions, 
insufficient  tolerances,  heat  damage,  etc.  Rate  the 
adequacy  of  system/component  design 
characteristics. 

LTM  106  -  Interoperability 

Interoperability  refers  to  the  condition  where  SE  of 
other  military  services  and  allied  forces  must  be 
compatible  with  the  aircraft  and  its  SE.  For  example, 
there  is  a  specific  test  requirement  for  U.S.  Army 
field  refueling  vehicles  to  both  accept  fuel  from  the 
aircraft  and  to  provide  a  refueling  capability  to  the 
aircraft  at  small  austere  airfields.  This 
interoperability  will  be  tested  and  evaluated  during 
the  course  of  DT&E.  In  general,  other  situations  will 
arise  where  non- Air  Force  equipment  may  be  used  to 
service  and  repair  the  aircraft.  Rate  the  ability  of  the 
weapon  system  and  its  SE  to  utilize  aerospace 
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ground  equipment  (AGE)  of  other  military  services 
and  allied  forces. 

12.6.2  Measurement  Criteria 

Each  LTM  will  be  considered  satisfactory  when 
rated  5  or  higher  on  the  LTDS  and  there  are  no 
significant  negative  comments  noted.  Logistics  teset 
measures  rated  4  or  below,  or  having  significant 
negative  comments,  will  be  analyzed  in  accordance 
with  the  procedures  in  Analysis  Techniques  and 
Tools. 

12.6.3  Analysis  Techniques  and  Tools 

Logistics  test  measures  rated  4  or  below,  or  having 
significant  negative  comments,  will  be  analyzed  to 
determine  their  validity.  All  comments  determined 
to  be  the  result  of  a  test-unique  nature  will  be 
excluded  from  analysis.  All  comments  applicable  to 
a  production-representative  condition  will  be 
analyzed  to  determine  root  cause  of  the  deficiency. 
Data,  when  analyzed,  should  be  grouped  by  systems 
(i.e.,  electrical,  ECS,  aircraft  power  plants,  avionics, 
airframe,  etc.).  The  LTMS  database  will  be  used  as  a 
source  of  information  for  evaluations. 


12.7  Overall  Element  Analysis 

Overall  element  analysis  will  be  a  cumulative  effort 
by  all  three  disciplines.  Individual  results  and 
conclusions  reached  by  each  discipline  will  be 
combined  to  form  a  single  rating  conclusion  for  the 
ILS  element  based  on  mission  impact. 

In  order  to  make  an  accurate  assessment  of  the  DI 
element,  the  impacts  on  all  other  ILS  elements  have 
to  be  taken  into  consideration.  Maintenance 
planning  will  be  evaluated  for  each  2-  and  3 -digit 
WUC  for  each  of  the  three  disciplines  (R&M,  HSI, 
and  LT).  In  addition,  data  from  the  following 
elements  will  be  combined  to  evaluate  the  impact  of 
integrated  ILS  elements  on  Design  Interface: 

1 .  Manpower  &  Personnel  (M&P) 

2.  Support  Equipment  (SE) 

3.  Supply  Support  (SS) 

4.  Technical  Data  (TD) 

5.  Training  and  Training  Support  (TTS) 

6.  Computer  Resources  Support  (CRS) 

7.  Facilities  (FA) 

8.  Packaging,  handling,  storage,  and 
transportation  (PHS&  T) 
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13.0  REPORTING  PROCEDURES 

13.1  Introduction 

This  section  is  designed  to  provide  information  on 
logistics  test  reporting  whether  in  the  form  of  a  TR, 
technical  letter  report  (TLR),  or  test  and  evaluation 
results  sheet  (TERS).  There  are  two  reporting 
procedures  that  must  be  considered  and  followed. 
Figure  4  depicts  the  procedures  to  follow  as  it  applies 
to  your  test  project  It  is  important  to  know  what 
documentation  your  customer  expects  and  the 
historical  significance  of  the  test  program. 

13.2  Combined  Test  Force  (CTF) 
Reporting 

Guidelines  have  generally  been  established  on 
reporting  T&E  results  within  each  specific  CTF. 
Check  with  your  test  project  manager  on  the 
procedures  for  reporting  logistics  test  results.  Your 
test  information  may  be  incorporated  into  a  single 
TR  published  by  the  CTF.  Participation  in  the  report 
coordination  meeting  is  a  must  to  ensure  your  data  is 
accurately  understood  and  reported.  Figure  4  depicts 
the  review  and  flow  of  this  information. 

13.3  Logistics  Test  Squadron  Reporting 

Guidelines  for  reporting  from  within  the 
organizations  vary  slightly  and  are  depleted  in  the 


Figure  5.  Coordination  and  reviews  within  the 
Logistics  Squadrons  should  occur  prior  a  technical 
review  is  scheduled. 

13.4  Assistance  in  Writing  Technical 
Reports 

AFFTC-TIH-97-01,  The  Author’s  Instruction  to 
Writing  AFFTC  Technical  Reports  (Reference  7), 
should  be  referred  to  for  help  in  writing  any  report. 
This  guide  can  be  obtained  from  412  Test  Wing 
Technical  Publications  Department. 
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Figure  4  Reporting  Procedures 
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Figure  5  Unit  Reporting  Procedures 
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LOGISTICS  TEST  DATA  SHEET  (LTDS) 

DT&E  □  OT&E  □  Contractor  □ 

Name: _  Employee  #: _  Years  Exp: _ 

Sex:  M  □  fD  Grade: _  AFSC: _  Date  (D/M/Y): _ 

Phone: _  DTIS#: _ LCN(1st4#s): _ 

JCN: _  Task: _ 

Evaluator  Review: _ Day  □  Night  □  Temp: _ F  Precip:  Yes  □  No  □ 


LOGISTICS  TEST  MEASURES  (LTM)  RATING  DEFINITIONS 

INSTRUCTIONS:  This  questionnaire  is  designed  to  assess  concerns  with  aircraft  maintenance.  The 
following  scale  will  be  used  for  your  evaluation: 


Very 

Unsatisfactory 

Unsatisfactory 

Marginally 

Unsatisfactory 

Marginally 

Satisfactory 

Satisfactory 

Very 

Satisfactory 

1 

2 

3 

4 

5 

6 

- Unsatisfactory  Ranqe - 

- Satisfactory  Range - 

Please  use  the  following  criteria  when  responding  to  the  attached  questions.  Respond  by  entering  the 
number  corresponding  to  the  scale  descriptions  ABOVE. 

N/A  -  Not  applicable 

VERY  UNSATISFACTORY  -  indicates  the  product  or  area  is  inadequate  and  requires  major 
improvements  such  as  redesign  or  rewrites. 

UNSATISFACTORY  -  indicates  the  product  or  area  is  inadequate  and  requires  improvements. 

MARGINALLY  UNSATISFACTORY  -  indicates  the  product  or  area  is  inadequate  and  minor 
improvements  are  required  to  make  it  more  helpful  to  maintenance  personnel. 

MARGINALLY  SATISFACTORY  -  indicates  although  the  product  or  area  is  adequate,  minor 
improvements  would  make  it  more  helpful  to  maintenance  personnel. 

SATISFACTORY  -  indicates  the  product  or  area  performs  its  intended  function. 

VERY  SATISFACTORY  -  indicates  there  is  little  room  for  improvement. 

Logistics  (L)  -  measures  weapon  system/subsystem  performance  and  supportability. 

Fluman  Factors  Engineering  (MSI)  -  measures  man-machine  interface. 

Environmental  (E)  -  measures  the  weapon  system/subsystem’s  effects  on  the  environment. 

Please  provide  comments  to  support  rating  after  each  question. 


DATABASE  USE  ONLY 

Entered  by  (initials) 

DR  Required? 

PCR  Required? 

Date  maintenance  notified: 
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1  -  MAINTENANCE  PLANNING  n/a  i  2  3  4  5  6 


LTM  10  -  Reliability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  actual  measured  reliability  characteristics  of  the  system,  and  their  relation  to  the 
established  and  planned  reliability  characteristics. 

LTM  11 -Maintainability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  actual  measured  maintainability  characteristics  of  the  system,  and  relation  to  the 
established  and  planned  maintainability  requirements. 

LTM  12 -Scope:  (L)  □  □□□□□□ 

Evaluate  task  coverage  in  relation  to  essential  versus  excessive  requirements. 

LTM  13  -  Frequency:  (L)  □  □□□□□□ 

How  adequate  was  the  frequency  with  which  the  inspection  or  task  was  performed. 

LTM  14 -Task Time:  (L)  □  □□□□□□ 

Evaluate  task  time  considering  all  unexpected  circumstances  that  increased  task  completion  time. 
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2  -  MANPOWER  AND  PERSONNEL  n/a  i  2  3  4  5  6 


LTM  20  -  Reliability:  (R&M)  □  □□□□□□ 

Subjectively  assess  changes  in  reliability  and  rate  the  relation  to  manpower  demands. 

LTM  21  -  Maintainability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  maintainability  and  its  relation  to  manpower  demands. 

LTM  22  -  Average  Crew  Size:  (R&M)  □  □□□□□□ 

Assess  the  average  number  of  personnel  required  to  perform  this  maintenance  task. 

LTM  23  -  AF  Specialty  Code:  (R&M)  □  □□□□□□ 

Rate  the  adequacy  of  a  specific  AF  maintenance  specialty  and  skill  level  used  to  accomplish  this  maintenance 
task. 

LTM  24  -  Human  Performance:  (HSl)  □  □□□□□□ 

Rate  the  ability  to  perform  the  task  in  all  environments  while  wearing  personnel  protective  equipment. 

LTM  25  -  Crew  Size  Adequacy:  (L)  □  □□□□□□ 

Evaluate  the  technical  order  data  (TOD)  specified  number  of  personnel  required  to  perform  the  task. 

LTM  26  -  AFSC  and  Skill  Level:  (L)  □  □□□□□□ 

Evaluate  the  TOD  identified  AFSC(s)  and  skill  level(s)  required  to  accomplish  the  maintenance  task. 
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Very 

Unsatisfactory 

Unsatisfactory 

Marginally 

Unsatisfactory 

Marginally 

Satisfactory 

Satisfactory 

Very 

Satisfactory 

1 

2 

3 

4 

5 

6 

3  -  SUPPORT  EQUIPMENT  (SE) 

N/A 

1 

2 

3 

4 

5 

6 

LTM  30  -  Reliability:  (R&M) 

Subjectively  assess  the  reliability  of  specific  support  equipment. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

LTM  31  -  Maintainability:  (R&M) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Subjectively  assess  the  maintainability  characteristics  of  the  support  equipment  relationship  to  established  and 
planned  maintainability  characteristics. 


LTM  32 -Ease  of  Use:  (L)  □  □□□□□□ 

Rate  the  adequacy  of  the  SE  in  relation  to  adequate  access  space  for  interfacing  SE  with  the  aircraft,  SE  visual 
and  other  communication  links  with  personnel,  efficient  arrangement  of  controls  and  displays,  and  compatibility 
with  personal  protective  equipment  under  various  environmental  conditions. 

LTM  33  -  Handling:  (L)  □  □□□□□□ 

Evaluate  the  SE  in  relation  to  handling/grasping  surfaces  provided  and  ease  of  transporting  (either  by  hand- 
carry  or  towing). 

LTM  34  -  Safety:  (HSl)  □  □□□□□□ 

Evaluate  the  SE  design  features  and  instructions  potential  to  minimizing  the  risk  of  personnel  injury  or 
equipment  damage  in  any  environment. 


LTM  35  -  Lunctionality:  (L)  □  □□□□□□ 

Evaluate  the  SE  in  relation  to  positioning,  adjustment,  and  performance  of  its  designated  function. 


LTM  36  -  Durability:  (L) 

Evaluate  the  SE’s  durability. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

LTM  37  -  Setup/Disassembly:  (L) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Evaluate  the  SE  in  relation  to  ease  of  maintenance  considering  setup  and  disassembly  times. 


Additional  Comments: 
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4  -  SUPPLY  SUPPORT  n/a  i  2  3  4  5  6 


LTM  40  -  Reliability:  (L)  □  □□□□□□ 

Rate  system  reliability,  are  the  planned  system  reliability  and  the  ILS  criteria  being  met 

LTM  41  -  SMR  Codes:  (L)  □  □□□□□□ 

Rate  the  appropriateness  of  the  SMR  code  identified  in  the  IPB  (when  performing  removal  tasks  on  equipment 
and  repair  tasks  off  equipment). 

LTM  42  -  Availability:  (L)  □  □□□□□□ 

Rate  the  availability  of  spares,  consumable  bench  stock  items,  and  suitable-subs  through  the  supply  system. 

Additional  Comments: _ 
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5  -  TECHNICAL  ORDER/DATA  n/a  i  2  3  4  5  6 


LTM  50  -  Reliability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  actual  maintenance  frequency  in  comparison  to  predicted  frequency  contained  in  LSAR. 

LTM  51  -  Maintainability:  (R&M)  □  □□□□□□ 

Subjectively  assess  evaluation  of  the  actual  task  time  and  crew  size  required  to  perform  the  maintenance  task  in 
comparison  to  the  values  contained  in  the  technical  data. 

LTM  52  -  Safety:  (HSI)  □  □□□□□□ 

Evaluate  the  technical  data  procedures  for  proper  placement  of  warnings,  cautions  and  notes. 

LTM  53  -  Effectiveness  of  Instrnctions:  (HSI)  □  □□□□□□ 

Evaluate  the  technical  data  instructions  for  simplicity,  accuracy,  and  clarity. 

LTM  54  -  Task  Requirement:  (L)  □  □□□□□□ 

Rate  (a)  the  existence  of  required  procedures  and  (b)  the  adequacy  of  the  identification  of  task  related 
requirements  (i.e.,  pre-existing  conditions,  proper  tools,  support  equipment,  consumables,  AFSC(s),  skill 
level(s),  and  other  technical  orders  for  use  during  the  same  task). 

LTM  55  -  Reference  Material:  (L)  □  □□□□□□ 

Evaluate  the  drawings,  figures,  and  schematie/wiring  diagrams  for  accuracy  and  value  as  a  fault  isolation  aid. 


Additional  Comments: 


■Unsatisfactory  Range' 


■Satisfactory  Range' 


6  -  TRAINING  AND  TRAINING  SUPPORT 

N/A 

1 

2 

3 

4 

5 

6 

LTM  60  -  Reliability:  (R&M) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Rate  the  actual  task  frequency  in  comparison  to  predicted  frequency  contained  in  LSAR. 

LTM  62  -  Safety:  (HF) 

Rate  the  adequacy  of  task  pertinent  safety  training. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

LTM  63  -  Knowledge  Training  (Type  1):  (L) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Evaluate  the  contractors  “theory  of  operation”  training  effectiveness  to  prepare  maintenance  personnel  for  OJT 
on  this  task. 


LTM  64  -  Knowledge  Training  (Type  4):  (L)  □  □□□□□□ 

Evaluate  the  Field  Training  Detachment  “theory  of  operation”  training  effectiveness  to  prepare  maintenance 
personnel  for  OJT  on  this  task. 

LTM  65  -  Proficiency  Training:  (L)  □  □□□□□□ 

Evaluate  the  OJT  training  received  for  this  task  (contractor,  FTD,  cross-utilization  training  CUT”,  and  system 
upgrade). 


Additional  Comments: 
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7  -  COMPUTER  RESOURCES  n/a  i  2  3  4  5  6 


LTM  70  -  CRS  Reliability:  (R&M)  □  □□□□□□ 

Rate  the  reliability  of  the  CRS  system  against  planned  parameters. 

LTM  71  -  CRS  Maintainability:  (R&M)  □  □□□□□□ 

Rate  the  maintainability  of  the  CRS  system  against  the  MTBM,  MTBMC,  and  MTTR  parameters. 

LTM  72  -  Diagnostic  System  Adequacy:  (BIT)  □  □□□□□□ 

Rate  the  adequacy  of  the  diagnostics  system  to  determine  fault  reporting  and  troubleshooting  support.. 

LTM  73 -Ease  of  Use:  (HSI)  □  □□□□□□ 

Rate  the  adequacy  of  the  BIT  system  as  a  fault  isolation  aid. 

LTM  74  -  Functional  Utility:  (L)  □  □□□□□□ 

Rate  the  adequacy  of  air  vehicle  automated/initiated  diagnostics  in  relation  to  accurate  and  efficient  fault 
detection  reporting,  isolation,  recording  and  correlating  of  malfunctions. 

LTM  75  -  Resource  Effectiveness:  (L)  □  □□□□□□ 

Rate  the  adequacy  of  the  hardware  and  software  utilized  to  upload  and  download  data  and  information  from 
various  systems,  both  on-  and  off-equipment. 


Additional  Comments: 


■Unsatisfactory  Range' 


■Satisfactory  Range' 


8  -  FACILITIES  n/a  i  2  3  4  5  6 

LTM  80  -  Reliability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  reliability  of  facility  systems  that  are  required  to  correctly  perform  maintenance. 

LTM  81  -  Maintainability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  maintainability  of  facilities  required  to  correctly  perform  the  maintenance  task. 

LTM  82  -  Safety:  (HF)  □  □□□□□□ 

Evaluate  the  facilities  safety  provisions  for  fire  protection  equipment,  ease  of  exit  and  entry,  sufficient 
ventilation  and  sufficient  emergency  equipment  (eyewashes,  first  aid  kits). 

LTM  83  -  Utilities:  (L)  □  □□□□□□ 

Rate  the  ability  of  the  site  utilities  (high  pressure  air,  temperature  and  humidity  control,  lighting,  electrical,  etc.) 
to  support  the  air  vehicle  and  support  equipment. 

LTM  84  -  Capacity:  (L)  □  □□□□□□ 

Evaluate  test  facility  size  and  layout  for  ease  of  aircraft  movement,  ability  to  utilize  SE  around  the  aircraft,  and 
equipment  storage  space  (including  both  SE  and  parts). 


Additional  Comments: 
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9  -  PHS&T  N/A  1  2  3  4  5  6 


LTM  90  -  Reliability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  PHS&T  influence  on  system  reliability  characteristics. 

LTM  91  -  Safety:  (HSI)  □  □□□□□□ 

Subjectively  assess  the  PHS&T  influence  on  system  reliability  characteristics. 

LTM  92  -  Suitability:  (L)  □  □□□□□□ 

Evaluate  the  packaging  materials  and  procedures  used  to  prevent  damage  to  parts  and  equipment  during 
shipment  and  storage. 

Comments: _ 


1  1 

I  2 

3 

I  4  I 

I _ 5 _ I _ 

_ 6 _ I 

I  - Unsatisfactory  Range -  I 

I  - Satisfactory  Ranqe -  I 

10- DESIGN  INTERFACE  n/a  i  2  3  4  5  6 

LTM  100  -  Reliability:  (R&M)  □  □□□□□□ 

Subjectively  assess  the  system’s  ability  to  operate  as  intended  under  the  defined  operational  and  support 
concept. 

LTM  101  -  Maintainability:  (R&M)  □  □□□□□□ 

Subjectively  evaluate  the  ability  of  an  item  to  be  retained  in  or  restored  to  specified  condition  when 
maintenance  is  performed  by  personnel  with  the  specified  skill  levels,  using  prescribed  procedures  and 
resources. 


LTM  102  -  Human/Machine  Interface:  (HSI) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Evaluate  the  air  vehicle  design  features  for  physical  and  visual 

access 

of  components. 

LTM  103  -  Safety:  (HSI) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Evaluate  the  design  features  used  to  minimize  risk  of  personal  injury  and  equipment  damage. 

LTM  104  -  Machine-Machine  Interface:  (L) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Evaluate  the  relative  positioning  of  LRUs  or  LRMs  for  unhindered  operation  and  maintenance. 

LTM  105  -  Design:  (L) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Evaluate  system/component  design  characteristics. 

LTM  106  -  Interoperability:  (L) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Rate  the  ability  of  the  weapon  system  and  its  support  equipment  to  utilize  aerospace  ground  equipment  (AGE) 
of  other  military  services  and  allied  forces. 


Additional  Comments: 
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PARAMETERS  AND  METHODS  OF  CALCULATION 
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RELIABILITY,  MAINTAINABILITY,  AND  AVAILABILITY  (RM&A) 
PARAMETERS  AND  METHODS  OF  CALCULATION 

INTRODUCTION 

This  document  describes  the  quantitative  RM&A  parameters  which  will  be  evaluated  during  developmental  test 
and  evaluation  (DT&E).  It  includes  a  detailed  description  of  the  terms,  definitions  and  formulas  for  each 
RM&A  parameter.  The  data  sources  for  each  parameter  in  the  calculations  is  also  provided. 

RELIABILITY 

MTBM(I)  -  Mean  Time  Between  Maintenance  (Inherent) 

Definition:  MTBM(I)  measures  the  mean  time  between  unscheduled,  on-equipment  maintenance  actions 
caused  by  a  design  or  manufacturing  defect,  with  time  expressed  as  total  production  aircraft  flight  hours.  This 
measure  includes: 

•  chargeable  inherent  maintenance  actions 

•  unscheduled  maintenance 

•  on-equipment  (line  or  organization  level) 

•  time  in  flying  hours  (production  aircraft). 

Exclude:  Test-unique  actions. 

Formula  1 : 


MTBM(I)  = 


TOTAL  FLIGHT  HOURS 
TOTAL  INHERENT  MAINTENANCE  ACTIONS 


Question:  What  are  test-unique  actions? 

Answer:  Actions  to  support  engineering  evaluations,  modifications,  etc.  Maintenance  actions  that  are  in  direct 
support  of  engineering  test  and  evaluation  of  subsystem  or  component  modifications,  technical  order  (T.O.) 
validation,  or  support  equipment  (SE)  compatibility  demonstration  will  be  considered  nonrelevant  to  both  the 
contractual  and  operational  RM&A  evaluation. 

Question:  What  are  inherent  maintenance  actions? 

Answer:  Failures  which  result  from  an  internal  cause.  These  failures  are  the  result  of  defective  design  or 
manufacture  and  will  be  coded  with  a  Type  =  1  How  Malfunction  Code  (HMC)  and  the  appropriate  Action 
Taken  Code  (ATK).  If  a  removed  component  is  later  found  to  be  serviceable,  the  action  will  be  changed  to  a 
NQ-DEFECT  maintenance  action.  There  will  be  a  maximum  of  one  chargeable  maintenance  action  with  an 
ATK  G  and  Type  =  1  HMC  combination  for  a  given  work  unit  code  (WUC)  within  a  given  job  control  number 
(JCN). 

Sources: 


•  Total  Flying  Hours:  Combined  Test  Force  (CTF)  Maintenance  Debrief  Form  207,  R&M 
cyclic  database. 

•  Total  Inherent  Maintenance  Actions:  System  Effectiveness  Data  System  (SEDS)  Failure 
Summary  Report. 
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MTBM(C)  -  Mean  Time  Between  Maintenance  (Total  Corrective) 

Definition:  MTBM(C)  measures  the  mean  time  between  unscheduled,  on-equipment  corrective  maintenance 
actions,  with  time  expressed  as  total  production  aircraft  flight  hours.  The  number  of  corrective  maintenance 
actions  is  the  sum  of  the  number  of  inherent,  induced  and  no  defect  maintenance  actions.  This  measure 
includes: 


•  corrective  maintenance  actions  (sum  of  inherent,  induced,  no  defect) 

•  unscheduled  maintenance 

•  time  in  flying  hours  (production  aircraft). 

•  on  equipment. 

Formula  2: 


MTBM(C)  = 


_ TOTAL  FLIGHT  HOURS _ 

TOTAL  CORRECTIVE  MAINTENANCE  ACTIONS 


Question:  What  are  induced  maintenance  actions? 

Answer:  Failures  which  are  caused  by  an  external  condition  and  not  by  an  internal  failure  pattern.  These 
actions  are  coded  with  a  Type  =  2  HMC  and  the  appropriate  ATK.  If  a  component  removed  due  to  an  induced 
failure  is  later  found  to  be  serviceable,  the  action  taken  will  be  changed  to  a  NO-DEFECT  maintenance  action. 
There  will  be  a  maximum  of  one  chargeable  maintenance  action  with  an  ATK  G  and  Type  =  2  HMC 
combination  for  a  given  WUC  within  a  given  JCN. 

Question:  What  are  no-defect  maintenance  actions? 

Answer:  Failures  which  result  from  indicated  malfunctions,  which  later  can  not  be  duplicated  or  components 
removed  to  facilitate  others  maintenance.  Time  compliance  technical  order  (TCTO)  and  scheduled  maintenance 
tasks  are  considered  support  general  and  are  not  included  in  this  category. 

Sources: 


•  Total  Flying  Hours:  CTF  Maintenance  Debrief  Form  207,  R&M  cyclic  database. 

•  Total  Corrective  Maintenance  Actions:  SEDS  Failure  Summary  Report.  This  is  the  sum  of  the 
inherent,  induced,  and  no-defect  actions. 

MTBR  -  Mean  Time  Between  Removal 

Definition:  MTBR  measures  the  mean  time  interval  between  removal  of  repairable  components,  with  time 
expressed  as  total  production  aircraft  flying  hours.  This  measure  includes: 

•  production  aircraft  flying  hours 

•  includes  indicated  failures  (retest  QK  [RTQK]) 

•  chargeable  removals 

-  on  equipment 

-  repairable. 


Exclude: 

-  removals  to  facilitate  other  maintenance 
- TCTQs 

-  scheduled  maintenance 

-  nonrepairable  components. 
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Formula  3: 


MTBR  = 


TOTAL  FLIGHT  HOURS 
TOTAL  NUMBER  OF  REMOVALS 


Sources: 


•  Total  Flying  Hours:  CTF  Maintenance  Debrief  Form  207,  R&M  cyclic  database. 

•  Total  Removals:  Retrieval  on  work  field  1  from  SEDS  database. 

MCSP  -  Mission  Completion  Success  Probability 

Definition:  MCSP  is  the  probability  that  a  scheduled  mission  will  be  completed  without  experiencing  an  on- 
equipment  failure  or  performance  degradation  that  results  in  the  aircraft  being  unable  to  complete  the  planned 
mission  objectives. 

Because  of  the  test  unique  environment  of  DT&E/IOT&E  an  analytic  model  will  be  used  to  evaluate  MCSP 
performance.  The  system  program  office  (SPO)  may  use  the  contractor  developed  model  (Probabilistic  Monte 
Carlo  simulation)  and  HQ  AFOTEC  may  use  its  own  model  (LCOM).  The  CTF  R&M  engineers  may  not 
evaluate  MCSP.  The  CTF  contribution  will  be  to  provide  data  to  HQ  AFOTEC  which  they  will  use  to  calculate 
MCSP.  This  approach  may  be  approved  by  the  JRMET  (Joint  Reliability  and  Maintainability  Evaluation 
Team). 

MAINTAINABILITY 

DMMH/FH  -  Direct  Maintenance  Man-hours  per  Flight  Hour 

Definition:  DMMH/FH  measures  the  number  of  man-hours  expended  by  maintenance  in  direct  support  of  the 
air  vehicle  and  its  engine(s)  per  aircraft  flight  hour.  This  measure  includes: 

•  scheduled  maintenanee 

•  unscheduled  maintenanee 

•  organizational  level  maintenance  man-hours. 

•  intermediate  level  maintenance  man-hours. 

Exclude:  test  unique  tasks  such  as: 

-  Instrumentation  maintenance 

-  deferred  manufacturing,  and  system  modification. 

Formula  3: 


DMMH  /  FH 


TOTAL  CHARGEABLE  MAN-HOURS 
TOTAL  FLYING  HOURS 


Sources: 


•  Total  Maintenance  Man-Hours:  From  SEDS  Active  Man-Hour  Task  Summary  (AMTS),  Chart  4. 

•  Total  Flight  Hour:  CTF  Maintenance  Debrief  Form  207,  R&M  cyclic  database. 

MMTR  -  Mean  Man-Hours  to  Repair 

Definition:  MMTR  measures  the  mean  time  required  to  complete  an  unscheduled  maintenance  action.  This 
includes: 


•  corrective  man-hours 

•  organizational  level 

•  intermediate  level. 
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Exclude: 


-  TCTO 

-  nonchargeable  actions. 
Formula  4: 


TOTAL  CORRECTIVE  MAINTENANCE  MAN-HOURS 

MMTR  = - 

TOTAL  CORRECTIVE  MAINTENANCE  ACTIONS 


Sources: 


•  Total  Corrective  Maintenance  Man-Hours:  SEDS  AMTS. 

•  Total  Corrective  Maintenance  Actions:  SEDS  Failure  Summary.  This  is  a  sum  of  the  Type  =  1,  Type  = 
2,  and  Type  =  6  maintenance  actions. 

BIT  -  Built-In-Test 

Definition:  Bit  adequacy  is  the  ability  to  accurately  detect  and  isolate  malfunctions  without  false  indications. 

•  BIT  Detection  (BIT-D)  -  The  percent  of  occurrences  in  which  BIT  correctly  detects  a  malfunction. 

•  BIT  Isolation  (BlT-1)  -  The  percent  of  occurrences  in  which  BIT  correctly  isolates  a  detected 
malfunction  to  the  failed  LRU. 

•  BIT  False  Indication  (BIT-Fl)  -  The  percent  of  occurrences  in  which  BIT  indicated  a  malfunction 
when  none  existed  (can  not  duplicate  [CND]  or  retest  OK  [RTOK]). 

Question:  What  is  an  occurrence? 

Answer:  An  occurrence  is  an  indication  of  an  aircraft  fault  by  the  BIT  system.  Every  maintenance  action  will 
be  documented  on  an  AFSC  Form  258  and  the  appropriate  BIT  code  will  be  recorded  as  an  occurrence.  The 
following  code  combinations  will  be  used  to  calculate  the  BIT  measures. 

BIT  Codes: 

#1:  BIT  detected  a  malfunction  when  one  existed. 

#2:  BIT  failed  to  detect  a  malfunction  when  one  existed  (BIT  was  designed  to  detect  this  malfunction). 

#3:  BIT  correctly  isolated  to  the  failed  LRU. 

#4:  BIT  did  not  isolate  to  the  failed  LRU. 

#5:  BIT  indicated  a  malfunction  when  none  existed  (verified  by  CND). 

#6:  BIT  indicated  a  malfunction  when  none  existed  (verified  by  RTOK). 

#7:  BIT  not  applicable. 

The  following  characters  will  follow  either  of  the  3  and  4  BIT  codes  above  described.  These  characters  reflect 
the  techniques  performed  during  the  accomplishment  of  a  maintenance  task: 

A.  BIT  and  Fault  Isolation  Manual  correctly  isolated  to  the  failed  LRU 

B.  BIT  and/or  Fault  Isolation  Manual  assisted  in  the  isolation  process.  Technical  orders  and  SE  were  also 

used  to  correctly  isolate  to  a  failed  LRU  (or  the  LRU's  wiring,  software,  etc.). 

C.  The  Fault  Isolation  Manual  did  not  isolate  to  the  malfunction  without  other  T.O.s  (such  as  wiring 

diagrams)  and/or  SE.  Primary  source  of  troubleshooting  was  NOT  the  Fault  Isolation  Manual. 

D.  Contractor  assisted  or  provided  isolation  or  fix.  (Established  T.O.  procedures  have  not  been  effective) 
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Formula  5: 


BIT  DETECTION  RATE  = 


Id) 
I<l  +  2) 


Formula  6: 


BIT  ISOLATION  RATE  = 


1(3) 

1(3+4) 


Formula  7: 


NOTES: 


•  BIT  Code  3  A  would  include  =  Code  3  +  Code  A 

•  BIT  Code  3B  would  include  =  Code  3  +  Code  B 

•  BIT  Code  4C  would  include  =  Code  4  +  Code  C 


Sources: 


The  source  of  the  BIT  codes  is  in  block  24  of  the  AFSC  Form  258.  The  BIT  code  data  will  be  extracted  from 
the  SEDS  database  and  manipulated  as  appropriate. 

LTM  72  -  Diagnostics  System  Adequacy 

An  analysis  of  the  BIT  system  adequacy  will  be  performed  to  determine  the  maintainability  capabilities  of  the 
aircraft.  BIT  usually  serves  two  functions:  (1)  to  monitor  and  report  system  status  to  an  operator;  and  (2)  to  be 
used  as  a  maintenance  tool  for  system  diagnostics  and  repairs.  The  objective  of  the  assessment  will  be  to 
determine  the  adequacy  of  the  BIT  system  as  a  tool  for  conducting  diagnostics  and  repairs.  That  is,  its 
capability  to  confirm  faults,  locate  and  isolate  the  cause  of  the  fault,  and  perform  repairs.  BIT  factors  of 
measures  of  false  alarms  (FA),  percent  of  correct  detection  given  a  fault  has  occurred  (Pcd),  percent  of  correct 
fault  isolation  (and  correct  fault  location)  given  correct  detection  (Pcfi),  and  mean  time  to  fault  locate  (MTTFL) 
will  be  used  to  determine  and  assess  the  adequacy  (i.e.,  system  effectiveness)  of  the  BIT  system. 

Measures  of  False  Alarms  (FA).  False  alarms  are  faults,  where  upon  investigation,  it  is  found  the  fault  cannot 
be  confirmed.  May  be  expressed  as  a  total  number,  percentage,  rate  of  occurrence,  probability  of  occurrence, 
etc. 

Formula  7:  (Percentage) 


Unconfirmed  Faults 

FA=  -  X  100 

Total  Number  of  Faults  Reported 


Percent  of  Correct  Detection  given  that  a  fault  has  occurred  (Pcd);  The  number  of  correct  detections  divided 
by  the  total  number  of  confirmed  faults  times  100  (to  express  the  quotient  as  a  percent). 
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Formula  8:  (Percentage) 


Number  of  Correct  Detections 

Pcd=  - X  100 

Total  Number  of  Confirmed  Faults 


Percent  of  Correct  Fault  Isolation  (and  correct  fault  location)  given  correct  detection  (Pcfi):  The  number  of 
correct  fault  isolations  (and/or  correct  fault  locations)  divided  by  the  number  of  correct  detections. 

Formula  9:  (Percentage) 


Number  of  Correct  Fault  Isolations 

Pcfi=  -  X  100 

Number  of  Correct  Detections 


Mean  Time  to  Fault  Locate  (MTTFL);  The  total  amount  of  time  required  to  locate  faults  divided  by  the  total 
number  of  faults. 

Formula  10:  (Mean) 


Total  Time  Reqd  to  Locate  Faults 

MTTFL  = . 

Total  Number  of  Faults 


AVAILABILITY 

Definition:  Availability  is  measured  in  terms  of  Full  Mission  Capable  rate,  Mission  Capable  rate,  and 
Maintenance  per  sortie.  Compliance  with  these  requirements  shall  be  determined  during  ISO  (initial  squadron 
operations)  and  ORE  (operational  readiness  evaluation).  ISO  will  begin  with  the  delivery  of  the  first  production 
aircraft  to  the  activation  site  and  end  with  IOC  (initial  operational  capability).  CTF  contribution  will  be  to 
provide  data  to  HQ  AFOTEC  which  they  will  use  to  calculate  the  availability. 
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APPENDIX  D 
LIST  OF  ACRONYMS 
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LIST  OF  ACRONYMS 


Abbreviation 

Definition 

AETC 

Air  Education  and  Training  Command 

AFFTC 

Air  Force  Flight  Test  Center 

API 

Air  Force  Instruction 

AFOTEC 

Air  Force  Operational  Test  and  Evaluation  Center 

AFSC 

Air  Force  Specialty  Code 

AFTO 

Air  Force  Technical  Order 

AGE 

aerospace  ground  equipment 

AMTS 

Active  Man-Hour  Task  Summary 

ATE 

automated  test  equipment 

ATK 

action  take  code 

AVS 

air  vehicle  specification 

Ao 

operational  availability 

BIT 

built-in-test 

CAMS 

core  automated  maintenance  system 

CFE 

contractor-furnished  equipment 

CMP 

component  discrepancy  report  analysis 

CND 

can  not  duplicate 

CRS 

computer  resources  support 

CTF 

combined  test  force 

CUT 

cross-utilization  training 

DAP 

data  analysis  plan 

DI 

design  interface 

DMMH/FH 

direct  maintenance  man-hours  per  flight  hour 

DR 

deficiency  report 

DT&E 

developmental  test  and  evaluation 

DTIS 

detailed  test  information  sheet 

ECS 

environmental  control  system 

ED 

engineering  data 

EFCS 

electronic  flight  control  system 

ET 

elapsed  time 

ExT 

excessive  time 

FA 

facilities/false  alarm 

FOM 

facilitate  other  maintenance 

FTP 

flight  test  plan 

GFE 

government-furnished  equipment 

HFE 

Human  Factors  Engineering 

HMC 

how  malfunction  code 

HSI 

human  systems  engineering 

IC 

integrated  circuit 

ILS 

integrated  Logistics  support 
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LIST  OF  ACRONYMS  (Continued) 


Abbreviation 

Definition 

ILSP 

Integrated  Logistics  Support  Plan 

IOC 

initial  operational  capability 

lOT&E 

initial  operational  test  and  evaluation 

IPB 

illustrated  parts  breakdown 

IPT 

Integrated  Product  Team 

ISD 

instructional  systems  development 

ISO 

initial  squadron  operations 

LCN 

logistics  control  number 

LRU 

line  replaceable  unit 

LSA 

Logistics  support  analysis 

LSAR 

Logistics  support  analysis  report 

LT&E 

Logistics  test  and  evaluation 

LTDS 

Logistics  test  data  sheet 

LAN 

local  area  network 

LRM 

line  replaceable  module 

JCN 

job  control  number 

JRMET 

joint  reliability  and  maintainability  evaluation  team 

LTM 

Logistics  test  measure 

M&P 

manpower  and  personnel 

MCSP 

mission  completion  suecess  probability 

MIP 

Material  Improvement  Program 

MMH 

maintenance  man-hours 

MMH/FH 

maintenance  man-hours  per  flight  hour 

MMTR 

mean  man-hours  to  repair 

MP 

maintenance  planning 

MTTFL 

mean  time  to  fault  locate 

MTBM 

mean  time  between  maintenance 

MTTR 

mean  time  to  repair 

MTBM(I) 

mean  time  between  maintenance  (inherent) 

MTBM(C) 

mean  time  between  maintenance  (total  corrective) 

MTBR 

mean  time  between  removal 

NATO 

North  Atlantic  Treaty  Organization 

O&M 

operations  and  maintenance 

OFP 

operational  flight  program 

OJT 

on-the-job  training 

ORE 

operational  readiness  evaluation 

OT&E 

operational  test  and  evaluation 

PCR 

publication  change  request 

Pcd 

percent  of  correct  detection 

Pcfi 

percent  of  correct  fault  isolation 
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LIST  OF  ACRONYMS  (Concluded) 


Abbreviation 

Definition 

R&M 

reliability  and  maintainability 

RCM 

reliability-centered  maintenance 

RDT&E 

research,  development,  test  and  evaluation 

RM&A 

reliability,  maintainability,  and  availability 

RTOK 

retest  OK 

SATCOM 

satellite  communication 

SE 

support  equipment 

SEDS 

System  Effectiveness  Data  System 

SPM 

system  program  manager 

SPO 

system  program  office 

SR 

service  reports 

SSSN 

system,  subsystem,  sub-subsytem  number 

ss 

supply  support 

SMR 

source,  maintenance,  and  recoverability 

SRU 

shop  replaceable  unit 

TD 

technical  data 

TEMP 

test  and  evaluation  master  plan 

TIS 

test  information  sheet 

TR 

technical  report 

TM 

technical  manual 

TTS 

test  and  training  support 

TCTO 

time  compliance  technical  order 

TLR 

technical  letter  report 

TERS 

test  and  evaluation  results  sheet 

T.O. 

technical  order 

TOD 

technical  order  data 

WIT 

watch  item 

WUC 

work  unit  code 

PHS&T 

packaging,  handling,  storage  and  transportation 
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AGARD  and  RTO  Flight  Test  Instrumentation  and  Flight  Test  Techniques  Series 

Volumes  in  the  AGARD  and  RTO  Flight  Test  Instrumentation  Series,  AGARDograph  160 

Volume 

Number 

1. 

Title 

Basic  Principles  of  Flight  Test  Instmmentation  Engineering  (Issue  2) 

Issue  1 ;  edited  by  A.  Pool  and  D.  Bosman 

Issue  2;  edited  by  R.  Borek  and  A.  Pool 

Publication 

Date 

1974 

1994 

2. 

In-Flight  Temperature  Measurements 
by  F.  Trenkle  and  M.  Reinhardt 
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3, 

The  Measurements  of  Fuel  Flow 
by  J.T.  France 

1972 

4. 
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by  M.  Vedrunes 

1973 

5, 
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11. 
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by  W.  Wuest 
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12. 

Aircraft  Flight  Test  Data  Processing  -  A  Review  of  the  State  of  the  Art 
by  L.J.  Smith  and  N.O.  Matthews 

1980 

13. 

Practical  Aspects  of  Instrumentation  System  Installation 
by  R.W.  Borek 

1981 

14. 

The  Analysis  of  Random  Data 
by  D.A.  Williams 

1981 

15. 
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by  B.  Stieler  and  FI.  Winter 

1982 

16. 
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1985 
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Publication 
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by  J.D.L.  Gregory 

1994 
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Reliability  and  Maintainability  Flight  Test  Techniques 
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